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PREFACE 



The Schools and Staffing Survey (SASS) was first administered in 1987-88 by the National 
Center for Education Statistics (NCES) to provide recurrent information on public and private 
elementary and secondary schools, teachers, and administrators, especially data on conditions 
affecting supply and demand for teachers and the characteristics of the teacher force. By linking 
survey data from classroom teachers and individual school sites to information from local education 
agencies (LEAs), the survey provides recurrent information on public and private elementary and 
secondary schools, teachers, and administrators to inform state and federal decisions, and the 
educational research community. The data now collected fall into four general categories: 



• Critical components of teacher supply, demand, and attrition, with attention to critical 
shortage areas and the policies and practices at all levels enacted to meet the demand 
in those areas 

• The professional characteristics, preparation, and experience of teachers and 
administrators, plus their perceptions of school conditions, professional 
responsibilities, decision making, and compensation policies 

• The conditions and characteristics of the school as a work place and a learning place, 
including characteristics of the student body, curriculum, special programs, and 
organizational structure 

• The implementation of school programs and policies such as English as a second 
language, bilingual education, diagnostic and prescriptive services, and programs for 
the gifted and talented 

The current survey design and process provide for a network of interlocking datasets from 
different organizational elements at the classroom, school, and district levels. The target population 
includes all elementary and secondary schools, teachers, and principals in the U.S. The process and 
instruments are as follows: 

1 . The School Administrator Survey and the School Survey are administered to a 
sample of 9,784 public and 3,360 private schools 

2. The Teacher Demand and Shortage Survey is administered to each sampled private 
school (embedded in the school questionnaire) and to the 5,500 LEAs of the sampled 
public schools 

3. The Teacher Survey is administered to a random sample of 56,736 public and 1 1,548 
private school teachers in the sampled schools 
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This process results in a comprehensive, linked database that provides national estimates for 
public and private schools, districts, principals, and teachers; state-level estimates for public data and 
affiliation-specific estimates for private data. 

SASS was administered at three-year intervals from 1987-88 through 1993-94, with a five 
year interval before its next administration in 1998-99. NCES is examining the direction, purposes, 
and uses for SASS in the twenty-first century. This includes scrutinizing the current uses of its data, 
its relationships with other federally sponsored data collection projects, and future national survey 
needs during a period of evolving policy priorities. 



As part of this process, NCES commissioned twelve papers to examine SASS and make 
recommendations about improving the scope and utility of the surveys. Authors were selected from 
the ranks of experts working to understand and describe the nation’s schools, and policymakers 
interested in instructional practice and professional development. They represent academia, the 
research community, and specialists in technology, teacher education, and state and local data 
collection. Authors were asked to examine the current SASS, address the effects of specific issues on 
future iterations of SASS, and make recommendations about improving, focusing, or expanding the 
scope and utility of the surveys. The papers were presented during seminars at NCES in the early 
part of 1996. 



The first paper, by Susan S. Stodolsky, addresses data collection on instructional practices and 
teaching effectiveness. Current reform movements embody an expectation that changes in teachers’ 
instructional practices will play an important role in improving student achievement, and that national 
measurement of those changes in classroom processes will track the progress of reform. Instruction 
and teaching effectiveness are central to the educational process and therefore, one might presume, 
also central to our collection of information about education. Stodolsky presents the argument for 
collecting data on instructional practices and teaching effectiveness; examines how teaching 
effectiveness is conceptualized; describes strengths and limitations of observational and survey data 
collection; suggests ways in which the effect of curricular reforms could be assessed; and proposes 
specific ways that SASS might measure instructional practice and content. 

In the next paper, David P. Baker suggests that SASS become a new organizational database 
for the nation’s K through 12th grade schools: an omnibus survey about the internal organization of 
elementary and secondary schools. Baker proposes that the foremost priority of SASS ought to be 
organizational and managerial information focusing on four main perspectives: school organization, 
multiple levels of governance, financial resources and flows, and school-level educational outcomes. 
Baker argues that if SASS were to become a central vehicle for NCES, it is essential to collect 



school-level data on educational outcomes such as student achievement, promotion, dropout, 
disciplinary actions, and college applications. SASS data could then contribute to the policy debate 
linking student outcomes with schools, their organizations, and resources. 

Some analysts suggest that computer use will completely transform classroom instruction in 
the next 20 years. In the third paper, Kathleen Fulton says that drastic changes are necessary in our 
data collection about technology if it is to keep pace with classroom innovations. While current data 
collection on classroom use of computers and related equipment may respond to public and 
congressional interest in defining the scope of computer use, it is primarily limited to numbers and 
availability of computers. Future data collection, she argues, needs to be refocused and directed 
toward defining the effect of state policies on access to technology in schools, how computers are 
actually used by teachers and students, and the effect of that use on teaching and learning. 

Phillip Kaufman argues that SASS data ought to link with information on student 
achievement. In his paper, Kaufman presents the feasibility and benefits of linking a student sample 
with SASS teacher and administrative data. He proposes that a successful merger of two data 
collection systems should produce data that could measure students’ overall academic performance, 
their growth in achievement, and their progress through critical transitions. A linkage between SASS 
and a student data component, Kaufman suggests, should also produce some administrative or 
respondent efficiencies and analytical benefits. 

Henry Y. Zheng’s paper discusses the scope and uses of the SASS School Administrator 
Questionnaire. He argues that current and future efforts to understand and guide educational reform 
will increase the importance of administrator survey data, especially such information as demographic 
and educational data, and information on principals’ attitudes toward school management issues such 
as the priorities of educational goals, seriousness of school problems, and the distribution of decision- 
making power in schools. Zheng suggests ways in which NCES might encourage greater use of the 
resulting data and recommends questionnaire modifications to increase data relevance. 

Dorothy M. Gilford’s paper addresses data collection on teachers’ inservice professional 
development. She proposes a framework with which to classify types of programs and discusses 
several current issues and their implications for professional development and data collection. Gilford 
recommends expanding the types of professional development items included in SASS; fielding a new 
computer coordinator survey; and eliminating the district survey by incorporating its essential 
questions into the principal survey. Gilford also notes that data collection must accommodate the 
current broad spectrum of professional development activities and their slow evolution from simple 



awareness programs designed to inform teachers about new ideas to more complex systemic programs 
shaped by constructivist principles and directed toward results-driven education. 

J. Michael Ross argues that the SASS sampling process ought to be redesigned to directly 
sample districts rather than schools. Given the importance of district-level data for systematically 
assessing the increased changes, complexities, and responsibilities in the organizational structures of 
schools and districts, Ross suggests that districts be sampled first, and then schools within the selected 
districts, a reversal on the current sample design. Redesigned district surveys should also de- 
emphasize teacher demand and supply issues to focus on district policy and reform information. Such 
changes, he suggests, would help NCES assemble important information that will be critical in 
assessing school reform. 

Rolf K. Blank recommends that the current design of SASS be linked with state and local 
education information systems to provide direct and important data on the characteristics of American 
schools and how education is carried out within them. Such a linkage, Blank asserts, would add to 
the usability and relevance of SASS data and increase state-level data analysis and reporting. He 
details three possible approaches and suggests NCES consider providing incentives for cooperation in 
state and local data collection. 

Jay G. Chambers also suggests a redesign of SASS to facilitate examination of resource 
allocation patterns in public and private schools. Chambers argues that such information would be 
valuable to researchers and other data users without unduly increasing respondent burden. Chambers’ 
paper and recommendations focus on personnel data since 80 percent of public school district budgets 
are devoted to personnel costs. Implementing his suggestions, he says, would provide a foundation 
tor addressing issues of equity, adequacy, and opportunity to learn within school systems. 

In a comprehensive examination of SASS, Erling E. Boe reconsiders the goals, foci, and 
strategy of SASS; the content balance, extent of coverage, redundancy of coverage, and potential new 
areas; and recommends data collection priorities. Boe recommends collecting data on both 
enduring" and "emerging" issues of policy concern. He suggests continuing to collect data in ten 
areas fundamental to the education process; continuing to collect data on the basic attributes of school 
principals, LEAs, and schools; and expanding data collection in eight areas of school 
governance/organization and instruction. The paper emphasizes public school data collection, and 
recommends that SASS data be made relevant to education policy development at all levels, since the 
mix of federal, state and local influences on schooling has been, and will continue to be, in flux. 




IV 



Susan P. Choy examines the depth of SASS, focusing on the level at which estimates should 
ybe provided, the respondent pool, and the response burden. Choy suggests that the relevance and 
importance of the original survey purposes remain intact, and the survey is able to capture 
information on enduring issues, even though changing policy concerns of the early 1990s have shifted 
the focus of some questions. Choy suggests that SASS monitor the extent to which various types of 
proposed reforms are actually present in schools and classrooms, and collect more information to 
describe what goes on at the classroom level. Choy also recommends continuing to collect data with 
which to provide state- and private school affiliation-level estimates. 

Finally, John Howard Burkett argues that there is a pressing public need for more state and 
local information on schools and that SASS must heed the public’s need for data. He echoes Boe’s 
call to focus on fundamental aspects of schooling that have been subject to major recent debates, 
policy action, or public concern. He suggests that the value of SASS will be realized only if it 
addresses education at the state and local levels. 

Individually and collectively, these papers set an ambitious agenda for NCES and SASS and 
provide the basis on which the Center can make decisions on how best to focus or expand the future 
direction and emphasis of SASS. 

In addition to the fine work of the authors, we also want to acknowledge the contributions of 
others that helped make this project a success. Within NCES, Sharon Bobbitt played a key role in 
conceptualizing the conference. When Sharon became Director of the Knowledge Applications 
Division in the Office of Educational Research and Improvement, Mary Rollefson ably guided the 
project to its conclusion. At Policy Studies Associates, John Mullens directed the project and was 
instrumental in bringing the conference and this publication to fruition. He was assisted by Eileen 
O’Brien, Janie Funkhouser, Amy Hightower, Ben Lagueruela, Kim Thomas, and Nancy Thornes. To 
each person, we extend our gratitude and appreciation. 



Paul Planchon 

Surveys and Cooperative Systems Group 
National Center for Education Statistics 



Dan Kasprzyk 

Education Surveys Program 

National Center for Education Statistics 
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SHOULD SASS MEASURE INSTRUCTIONAL PROCESSES 
AND TEACHER EFFECTIVENESS ? 1 

Susan S. Stodolsky, University of Chicago 



This paper addresses issues related to the 
possible inclusion of items in SASS to 
measure instructional practices and 
teaching effectiveness. In order to answer 
the question posed, the paper explores 
what purposes can be served by measuring 
instructional practices on a national scale. 

It then examines how teaching 
effectiveness is conceptualized. Turning 
next to substantive and methodological 
concerns, a section describes some 
strengths and limitations of observational 
studies of classroom instruction and makes 
a similar assessment of survey studies. 
Some attention is then given to curricular 
reforms and how their impact might be 
assessed. The last sections of the paper 
suggest specific ways in which 
instructional practices and curricular 
content might be measured through SASS, 
including the selection of specific school 
subjects and grade levels for attention. 

Why Collect National Data on 
Instruction? 

We begin by briefly examining some of 
the main reasons to collect information 
about instructional practices/processes on a 
national scale. If we are to understand, 
monitor, and improve our nation’s schools, 
accurate and timely empirical, descriptive 
data about how schools work must be 
available. The activities that take place in 
classrooms to engender student learning 
and development are the heart of any 



school’s educational efforts. It is in the 
transactions between and among teachers, 
students, materials and tasks that deliberate 
efforts to educate occur. 

Descriptive information about how 
teaching and learning occur in classrooms 
and about what is taught provides the basis 
for monitoring the status of instruction in a 
large number of settings. Such 
information can provide periodic 
assessments of stability and change in 
instruction, particularly as changes relate 
to deliberate efforts to reform or alter 
curriculum and instruction. Similarly, if 
collected along with knowledge of 
particular policy initiatives, curriculum 
standards, or changes in teacher 
preparation or staff development, 
information describing classroom 
instruction can help track the impact of 
various policies on what transpires in 
classrooms. 

Descriptive information about classroom 
processes also can contribute to the 
deliberations of teachers, teacher 
educators, subject matter and other 
educational associations, and policy 
makers at local, state, and national levels. 
Basic researchers also benefit from 
information about what actually goes on in 
classrooms. 

Possible Limitations 

While a national picture of instruction is 
desirable for the reasons mentioned, it can 



be argued that the SASS sampling strategy 
of few teachers in any school works 
against the utility of such information for 
local (school, district) policy makers. 

Many reform efforts are local and data on 
instruction collected within the current 
SASS sampling strategy might not be 
sensitive enough to detect local effects. 
These are cogent criticisms and suggest 
some alteration of the SASS sampling 
strategy to make data on instruction and 
the effects of reform even more useful. 
Nevertheless, instructional data that can be 
analyzed at national, state, and regional 
levels seems highly useful especially since 
many reform programs are conducted at 
these levels. We will return to sampling 
issues in the section on recommendations. 

Many argue that classroom process 
information is most valuable when 
connected to student achievement and 
attainment. Data from NELS have been 
analyzed recently by Kupermintz, Ennis, 
Hamilton, Talbert, and Snow (1995) and 
Lee and Smith (1995). Both research 
groups found significant relationships 
between certain measures of instructional 
practices (e.g., emphasis on higher order 
thinking), teacher attitudes (e.g., 
willingness to alter instructional practices 
if students are not learning) and student 
performance on both math knowledge 
(lower mental process) and math reasoning 
(higher mental process) items. A similar 
analysis of teachers’ responses to the 
CLAS survey by Wiley and Koon (1995) 
also demonstrates the potential for 
connections between instructional items 
and student attainment. 

Although NCES has considered monitoring 
student achievement in connection with 
SASS, as currently structured SASS is not 



linked to student data on achievement or 
other outcomes and a considerable 
redesign and change in sampling strategy 
would be required to do so. 2 The 
question then is whether information on 
instructional practices and content 
coverage is still useful in the absence of 
data on student attainment. We argue that 
links to achievement can be direct and 
empirical as when teachers and their 
students are studied, or links can be 
putative and conceptual based on known or 
assumed connections between practices and 
achievement. Existing research and theory 
can be used to formulate the presumptive 
connections and might also inform the 
design of empirical studies. 

For example, studies such as Kupermintz 
et al. (1995) and the IEA studies have 
shown that content coverage is related to 
student attainment as are certain 
instructional processes. While our 
knowledge is far from complete in this 
area, it seems safe to assume that content 
covered, particularly content that is 
emphasized, is more likely to be learned 
than topics not taught or emphasized 
during instruction. Thus, a description of 
instructional practices and content 
coverage would allow some inferences as 
to what students are likely to learn. More 
in-depth and direct measurement of both 
processes and student learning might be 
desirable, but including information on 
curriculum and instruction in SASS 
appears a useful first step. 

How Is Teaching Effectiveness 
Conceptualized? 

The charge for this paper includes a 
consideration of whether measures of 
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instructional practices and teaching 
effectiveness should be included in future 
SASS instruments. It must be noted here 
that a broad consensus on a definition of 
effective or good teaching does not exist. 
Empirical evidence, theory and values 
along with specified criteria for 
effectiveness all enter into a conception of 
effective teaching. In addition, 
considerable evidence that instructional 
practices need to be tailored to subject 
matter, developmental levels of students, 
and other factors is now available, 
suggesting that effectiveness comes in a 
number of varieties. 

Although not all teachers and policy 
makers endorse one view of effective 
teaching in a given time period, visions of 
effective teaching change over time. For 
instance, during the late 1970s and 1980s, 
the process-product research program 
(Brophy & Good, 1986) assumed there 
were generic characteristics of good 
teaching (i.e., they apply to all school 
subjects and grade levels considered). By 
focusing primarily on features of teacher- 
centered instruction, this influential, 
empirical research program identified a 
number of teacher behaviors (direct 
instruction model) that correlated with 
student gains on standardized achievement 
tests in reading and math, primarily in 
elementary schools. 

Critiques of the process-product view point 
out that the correlational method embodied 
the existential fallacy (Stodolsky, 1988). 
That is, only currently used practices could 
enter the model of effective teaching. The 
use of data on individual teacher behaviors 
decontextualized instruction and made it 
difficult to know how to put the 
instructional program into operation as a 



combination of student and teacher 
behaviors in an intact lesson structure. 

The model excluded behaviors that might 
be subject- or grade-level specific. The 
model did not examine student behaviors. 
The model adopted a transmission view of 
teaching. Effectiveness was 
operationalized by achievement tests that 
almost exclusively contained lower-mental 
process skill items. 

Now, as evident in many standards and 
reform documents, a constructivist point of 
view of learning and teaching is holding 
sway. This view directs attention to 
students’ active role in the learning 
process. Classroom arrangements such as 
group work, debate and discussion are 
believed integral to effective instruction. 

In addition, teaching and learning are 
assumed to be different from one school 
subject (or even topic) to another. No 
fully general model of effective teaching 
and learning is expected. Last, different 
student criteria are employed to judge 
effectiveness. More emphasis is placed on 
reasoning, problem solving, creative 
production and long-term products. 
Methods of student assessment beyond 
standardized achievement tests are 
endorsed. 

The transmission and constructivist views 
of teaching both may have a proper place 
in the analysis of teaching effectiveness. 
Flexibility in instructional strategies may 
be a hallmark of effective teaching. 
Different instructional practices may be 
desirable depending on instructional goals 
and lesson formats. Instruments to assess 
effective practices must contain an 
appropriate range of items to tap lesson 
structures, content, instructional strategies 
and teacher and student activities. There 



is a danger in only assessing popular 
visions of effective teaching which may 
not be widely implemented or universally 
appropriate at any given time. 

Some Features of Classroom 
Activity and Teacher Behavior: 
Observational Studies 

Observations of instructional activity are 
often thought to be the most valid method 
of data collection. While observational 
studies are generally beyond the scope of 
proposed SASS activities because they are 
very expensive, direct observation can 
often be used in early stages of instrument 
development to provide relevant categories 
and items for surveys and other 
measurement approaches. Observations 
also have a place in validity studies and in 
small-scale focussed research. 

Observational studies provide accumulated 
knowledge from which to formulate 
productive questions. Useful reviews of 
research on curriculum and teaching are 
provided by Shulman (1986) and Darling- 
Hammond and Snyder (1992). Here we 
take a selective look at past observational 
research on teacher behavior and 
classroom activity. 

Observational studies (e.g., Good & 
Brophy, 1986; Goodlad, 1984) have 
documented a robust picture of teacher- 
centered instruction primarily oriented 
toward lower-level cognitive goals. 
Recitations, variants on lecture, and 
seatwork are the primary instructional 
formats used in most classrooms. 

However, systematic variation occurs 
when subject matter is examined. 

Similarity of instructional practices across 



teachers may be greater in subjects such as 
mathematics, than in subjects such as 
social studies or English (Stodolsky, 

1988). Variation is also tied to teachers’ 
conceptions of subject matter and goals 
(Grossman & Stodolsky, 1994, 1995; 
Shulman, 1987). 

A variety of contextual and situational 
factors produce variation in teaching and 
consequently limit the stability and 
generalizability that can be expected in 
studies of teacher behavior, especially at 
the level of the individual teacher. As 
noted elsewhere (Stodolsky, 1990) subject 
matter, grade level, lesson type, and 
lesson goal account for variation in 
teaching behaviors and instructional 
arrangements. In addition, the type of 
students and track level of courses (Oakes, 
1985) along with district policies, type of 
school, and other institutional factors may 
all influence a teacher’s choice of 
curricular content and instructional 
methods. 

Limitations of Data from Observational 
Studies 

Perhaps one of the most important 
limitations of available observational 
studies, a feature shared with survey 
studies, is that the contexts studied are 
limited. The preponderance of large-scale 
observational studies have been conducted 
with elementary school teachers of reading 
and math. A few have focused on social 
studies instruction at the elementary and 
high school levels (Stodolsky, 1988; 
Newmann, 1992; Newmann & Wehlage, 
1995). While there are a variety of small- 
scale observational studies, including 
studies of classroom discourse, our 
knowledge is not deep with respect to the 
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state of classroom curriculum and 
instruction in fields such as science, social 
studies, English, foreign language or the 
arts. We also have surprisingly few 
observations of what actually takes place 
in high school classrooms. 

Another limitation of available 
observational research is that it has 
focused primarily on teacher behaviors. A 
more ecological approach to classroom 
settings, such as employed by Doyle 
(1983), Gump (1982), and Stodolsky 
(1988), examines classroom activities, and 
incorporates knowledge of what both 
students and teachers do during 
instruction, along with knowledge of 
materials and tasks. However, studies of 
classroom ecology have been relatively 
rare. In most observational studies, when 
students are observed it is to assess their 
on-task behavior or involvement. 

Nevertheless, observations can provide 
evidence of real instructional experiences 
unrivaled by other methods. Particularly 
if one wants to understand the qualities of 
transactions that occur in classrooms and 
their intellectual and social features, 
observations can play a possibly unique 
role. Observations, done properly, can 
reveal the connections between what is 
taught and how it is taught— observations 
can preserve classroom events as they 
occur together. The issue for NCES or 
others striving for a national picture of 
curriculum and instruction is under what 
circumstances, if any, direct observation 
should be used as a data gathering 
approach. Clearly, observations cannot be 
done of the large number of teachers 
currently surveyed by SASS. But some 
observational work may add to the validity 
and richness of the SASS enterprise. It is 



also important to determine the utility of 
information obtained with other methods 
such as teacher logs and surveys compared 
to observational data. 

Studies of Curriculum and 
Instructional Processes: Survey 
Research 

A number of large survey studies, often 
funded by NCES, NSF, and OERI, have 
provided valuable information about 
curriculum and instruction in our nation’s 
classrooms. The main contributors to our 
knowledge about curriculum and 
instruction on a national scale are 
NELS:88, NAEP, Reform Up Close 
(RUC) and SIMS. Weiss (1993) provides 
some useful information on instructional 
practices among math and science 
teachers. The validity of using surveys as 
a measurement tool in the area of 
curriculum and instruction has also been 
examined (Burstein et al., 1995; Porter, 
1995; TIMSS, 1994) and survey 
development is ongoing (Porter, 1995). 

Useful reviews and analysis of many of 
these survey projects can be found in 
NCES working papers (Leighton et al., 
1995), a report by Porter (1995), the work 
of Schmidt and McKnight (1995), and 
work by Policy Studies Associates 
(Leighton, 1994; Leighton & Mullens, 
1994; Leighton, Turnbull, & Mullens, 
1994; Mullens, Weiner, Williams, & 
Turnbull, 1994). A catalogue of 
instruments measuring the enacted 
curriculum in math and science at the 
middle and high school levels is now 
available (Porter & Smithson, 1995). A 
list of sources for major surveys can be 
found in the Appendix. 




1 K 



-a_ \J> 



5 



The surveys distinguish between plans 
(intended curriculum or objectives) and 
actions (enacted or implemented 
curriculum) with the latter emphasized. 

To varying degrees, these surveys seek to 
measure plans (instructional goals and 
desired outcomes), to document what is 
taught (content/ topics and intellectual 
processes, time allocations, emphases); 
how instruction is organized (pedagogy, 
teacher and student activity, homework 
and tests); and resource use (e.g., 
technology, textbooks). The surveys are 
often described as measures of students’ 
opportunity to learn (OTL), a term 
borrowed from the IEA studies. 
McDonnell (1995) provides a useful 
discussion of the OTL construct. 

The uneven coverage of contexts found in 
observational studies is also characteristic 
of the survey research. In an interesting 
juxtaposition, however, most of the 
surveys deal with high school or eighth- 
grade instruction while the observation 
studies are mainly at the elementary level. 
In fact, with the exception of NAEP 
fourth-grade surveys, Weiss (1994), the 
Consortium on Chicago School Reform 
(1994) which borrows from NELS and 
RUC, and the CRC (1994) survey of 
elementary math teachers in California, it 
was difficult to locate surveys of the 
enacted curriculum given on a large scale 
at the elementary level. High school 
coverage is also somewhat uneven. NAEP 
targets twelfth grade and therefore obtains 
information primarily about advanced 
courses. There is reason to believe that 
the practices used in more advanced 
courses may differ to some extent from 
those in the earlier years of high school. 
Burstein et al. (1995) document that 
teachers of more advanced courses are 



more accurate in reporting topic/content 
coverage and emphases. 

The surveys are also uneven with respect 
to subject matter coverage. Largely due to 
the efforts of NSF, major survey 
development has occurred in math and 
considerable attention has also been paid to 
science instruction. Applebee (1981, 1992) 
conducted national surveys on the teaching 
of writing and literature at the high school 
level which provide modest amounts of 
information about instructional practices; 
the Applebee work might be a starting 
point for further survey development in 
English along with available NELS items 
on English. According to Andy Porter, the 
CPRE School-Based Management Survey 
(SBM) also contains items dealing with 
instruction in language arts and social 
studies at the elementary and high school 
levels. The items follow the four-part 
scheme developed by Porter and others to 
assess teachers’ objectives, content 
covered, modes of instruction and 
cognitive processes. The content items in 
the CPRE surveys are rather general and 
might provide only a starting point for 
item development in English and social 
studies. 

It seems more than financial support has 
led to so much attention to curriculum and 
instruction measures in math. 

Mathematics lends itself to a systematic 
analysis of its content, topics, and 
operations because it is the best defined 
and probably least contentious of all school 
subjects. Compared to other subjects, 
there is considerable agreement among 
math teachers and teacher educators about 
best practice. 
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Mapping curriculum topics in other fields 
may pose a greater challenge than mapping 
topics in math. Our own work (Stodolsky 
& Grossman, 1995) on five academic 
subjects and an analysis of English by 
Grossman (1993) and Elbow (1990), 
suggest that there is less agreement about 
content and teaching methods in subjects 
such as English and social studies. The 
difficulties confronted in developing social 
studies and language arts curriculum 
standards, confirm the lack of consensus in 
these fields. Teachers of English and 
social studies expect considerable 
autonomy in the selection of course 
content, especially because they are not 
constrained by a perceived content 
sequence. Science teachers also report 
freedom in choice of topics, but share a 
commitment to the scientific method. The 
development of surveys with detailed 
topical analyses for English and social 
studies presents a challenge in curricular 
analysis and instrument development. 

In sum, significant recent efforts to 
develop surveys of enacted curriculum and 
instructional practices have not been 
undertaken at a level of effort similar to 
that in math and science in the fields of 
English, social sciences, foreign language, 
and other subjects including the arts. 
Considerable new survey development, 
particularly on curriculum topics, would 
be required to obtain information about 
instruction in a range of subjects and grade 
levels. 



Curricular Reforms 

Before moving on to specific suggestions 
regarding future directions for SASS, the 
issue of curricular reforms needs some 



discussion. Studies of teachers in settings 
in which reforms are under way have 
found a mixed picture at best. For 
example, in case studies of mathematics 
teachers attempting to implement the 
California Math Frameworks, Cohen and 
Peterson (1990) found only modest changes 
from conventional practice, confirming the 
suggestion by Burstein et al. (1995) that 
new practices are "layered" on to old 
ones. On the other hand, certain changes 
in math teaching such as the introduction 
of calculators seem more widespread 
(Weiss, 1994). 

Research on the Coalition for Essential 
Schools (Muncey & McQuillan, 1993; 

Little, 1995) documents great variety in the 
extent to which teachers adopt Coalition 
principles. However, many teachers 
report using process writing approaches 
such as those advocated by the National 
Writing Project (Freedman, 1987; NAEP 
Report Card on Writing). 

To help understand the implementation of 
reforms, a first step might be to learn what 
teachers actually know about proposed 
reforms and standards. Adequate teacher 
knowledge and understanding of reforms is 
far from guaranteed just because standards 
are published or new frameworks drawn 
up. The CRC survey 3 provides excellent 
examples of items used to assess teacher 
knowledge of curriculum frameworks. It 
would also be desirable to obtain 
knowledge of organizational support and 
provision of resources for reform in 
departments and schools. In addition, it is 
important to determine if teachers are 
asked to act simultaneously on a number 
of policy initiatives which may not be 
consistent with one another. 



If one of the purposes in monitoring 
instruction in the nation is to provide 
information about the progress of 
curricular reforms, it must be assured that 
the item pool used to measure curriculum 
and instruction is adequately tailored to the 
reforms advocated in each subject matter 
studied. 

An examination of the standards for 
curriculum in science (NRC, 1994; 
Rutherford & Ahlgren, 1990), social 
studies (NCSS, 1995), mathematics 
(NCTM, 1989, 1991) and 
English/Language arts (NCTE, 1996) 
suggests different degrees of emphasis on 
changing pedagogy and changing content. 
The math standards may be most explicit 
with respect to the vision they embody of 
pedagogy consistent with the recommended 
standards. 4 The use of open-ended and 
student-generated problems and 
investigations which take place over a 
number of days are examples of a 
constructivist pedagogy endorsed by 
NCTM. Specific items have been written 
to address features of pedagogy in the 
NCTM standards; the CRC survey has 
some excellent examples. Porter (1995) 
reports making use of the NCTM 
standards and NSTA standards in 
developing opportunity to learn topic items 
for math and science. 

Specialized terminology or language poses 
a possible problem in instrument 
development with items geared toward 
reforms. Burstein et al. (1995) in their 
validity study of math instruction items 
found that teachers did not always interpret 
terms in the same manner (e.g., "math 
modeling" had a number of different 
meanings to the teachers they studied). A 
term like "investigations" used in the 



NCTM and California math frameworks 
might carry a variety of connotations. 
Indeed, the term "reform" itself is not 
used equivalently by those reporting about 
it. 

Last, in some cases the new standards are 
predicated on teacher mastery of subject 
matter and pedagogical content knowledge 
not currently widely held in the teaching 
force. The TIMSS survey and the CRC 
survey for math teachers, include items to 
reveal teachers’ conceptual understanding 
of mathematical material along with 
pedagogy. It seems likely that items of 
this type would predict student attainment, 
and help us document barriers to 
implementation of reforms. As such, they 
seem important to include in any effort to 
measure curricular reform. 



Should SASS Include Measures of 
Instructional Practice? 

Except for NAEP, there does not appear to 
be any federal program in which 
instructional practices and opportunity to 
learn will be monitored in the future. 

SASS, with its large sample of teachers, 
seems an excellent vehicle for the 
measurement of curriculum and 
instructional practices. However, the 
inclusion of a fairly comprehensive set of 
items on content (e.g., as in TIMSS or the 
Porter OTL four-dimension scheme) would 
involve a lot of additional respondent time. 
Further, to adequately monitor pedagogy 
and track reforms, additional items would 
be needed. 

Since NAEP is an ongoing program that 
taps into curriculum and instruction in a 
number of school subjects (although maybe 
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not very deeply), an optimal plan for 
SASS would complement and supplement 
efforts planned under NAEP. Some 
school subjects and grade levels not 
regularly covered by NAEP should be 
included in SASS. At the same time, 
more targeted efforts to link with NAEP 
and/or assist NAEP to enhance its 
curriculum and instruction measures would 
be highly desirable. Some links to NAEP 
would also provide tie-ins to student 
survey responses about their instructional 
experiences. 

SASS seems ideally suited to monitor the 
classroom consequences of reforms such as 
curriculum standards. (It may be asking 
too much to monitor the myriad of other 
reforms under way.) A selection of 
specific school subjects and grade levels 
seems the best strategy here. However, in 
order to maximize insight into how 
reforms work, it would be desirable to 
have more teacher respondents from a 
given school than has been the case in 
previous SASS sampling, so that 
information about the presence of 
particular reform efforts in the schools 
could be obtained. Linking with NAEP 
under selected circumstances would also 
benefit from more clustering of teachers in 
schools. 



Item Selection 

Curriculum Content 

Let us begin by examining measures of 
content taught. As discussed earlier, there 
is a substantial pool of items to use in 
measuring the content taught in science 
and mathematics, especially at the middle 
school and high school levels. Limited 



topical analysis is also available in U.S. 
and world history, although not the 
broader social studies. High school English 
is not mapped in much detail nor is the 
elementary school curriculum. (Exceptions 
are the three-dimensional content structure 
developed by Freeman, Porter and others 
for fourth-grade mathematics and some 
items from NAEP dealing with reading 
and writing instruction). The four- 
dimension topic items such as developed 
by Porter (1995) for his recent OTL study 
for math and science, seem a suitable 
model for item sets to be used in SASS. 

The four dimensions include two 
dimensions of topics and the degree of 
emphasis each receives, cognitive activities 
(with time distribution) and the medium 
(mode) of instruction (with time 
distribution). For school subjects other 
than math and science, item development 
analogous to the Porter model would be 
needed. 

Pedagogy 

There are quite a few items and item types 
dealing with pedagogy or teaching methods 
that seem applicable to most subjects and 
grade levels, although a careful analysis 
would be required to assure that practices 
found in elementary school classrooms 
were adequately sampled. The language in 
which methods are described might also 
require modification and field testing when 
applied in contexts other than those 
previously surveyed. In addition, 
specialized language from reform 
documents should be used with caution and 
fully pilot tested to assure common 
understandings. 

As Leighton, Turnbull, and Mullens (1994) 
note, subject-specific questionnaires have 



been the rule recently. Many common 
instructional items reappear in surveys for 
teachers of different subjects in addition to 
specific items for each subject. The 1994- 
95 SASS Follow-up Teacher Questionnaire 
has a number of sections dealing with 
teaching methods that are promising and 
which build on development work from 
other surveys we have discussed. While a 
good starting point, a careful review 
should be made for appropriateness to 
grade levels and school subjects selected 
for study. Also, there may be some 
overlap in constructs if four-dimension 
content items such as those in Porter are 
also in the survey. 

Goals 

Burstein et al. (1995) recommend against 
the inclusion of items measuring goals, as 
they did not find a good match with 
responses and other data sources such as 
the goals inferred in tests or teacher 
assignments. On the other hand, they did 
find meaningful relationships between 
endorsement of reform goals and reform 
practices, but not between endorsement of 
traditional goals and traditional practices. 
This issue would seem to require further 
study before eliminating goal items from 
national surveys. The data pattern 
suggests in part that most teachers believe 
traditional goals are worthwhile, even 
those who are moving their practice in the 
direction of reform. This finding seems 
another example of the tendency of 
teachers to add on to their practice without 
giving up old patterns. Thus, some 
tensions inherent in change may be 
revealed effectively through analysis of 
goal items. Although not the highest 
priority, if respondent time allows, goal 
items should be retained. 



Items to Track Curricular Reforms 

In addition to content/ topic items and 
pedagogy items, new items should be 
developed that assess teachers’ specific 
knowledge of reforms. The CRC survey 
provides some good examples of such 
items for the California Math Frameworks. 
Teacher’s subject matter and pedagogical 
content knowledge required for 
implementing reform should also be 
measured. 5 A particularly promising item 
format has been used in TIMSS and the 
CRC survey, among others. The items 
ask teachers to envision an instructional 
sequence of lesson parts used to teach a 
specified topic. For example, the CRC 
survey asked questions about instruction 
dealing with fractions in an open-response 
format. The TIMMS items are more 
structured. These items tap lesson 
organization, content emphasis, 
pedagogical content knowledge and subject 
matter knowledge and may be an effective 
way to tie together features of instructional 
processes and content in a manner that 
approximates what actually happens in 
classrooms. 

Teacher Attitudes, Professional Activities, 
and School Culture 

In creating item sets for a survey, it would 
be desirable to include measures of teacher 
efficacy and willingness to adapt 
instruction as these scales have important 
predictive power in connection with other 
instructional items. Professional 
development activities and participation in 
subject area and other networks should 
also be assessed. Items that assess the 
extent to which the school culture and 
organization support reform are also 
useful. Basic information such as whether 
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a school or department has officially 
adopted a particular reform should be 
collected. Taken together, these items 
would reflect teachers’ opportunities to 
learn about or deepen understanding of 
new approaches and to gain support in 
trying to implement reforms. These scales 
could be part of the teacher background 
section of the survey. 

To create respondent time for the 
suggested content/pedagogy and teacher 
knowledge items, we suggest two 
strategies. One is to eliminate certain 
parts of the current SASS survey since 
items have been administered over many 
years and may be given to a subsample or 
less frequently. In particular, items 
dealing with teacher control over policies 
such as discipline, hiring of new teachers 
could be omitted. The list of perceived 
problems (poverty, tardiness, etc.) might 
also be eliminated or given to a subsample 
of teachers. The second strategy takes us 
into the realm of sampling to which we 
now turn. 



Who Should Be Surveyed? 

We believe that all SASS respondents need 
not answer all survey items. We 
recommend the use of item/person 
sampling in the administration of SASS. 
We recommend selecting teachers of 
certain grades and school subjects to 
respond to the curriculum and instruction 
survey. Other respondents could be used 
to answer more general questions from 
SASS. In addition, even teachers within 
the recommended grades and subjects 
could be directed (say, by use of their 
birthday as a sorting mechanism) to 
answer only certain parts of the survey. 



While targeting teachers of certain subjects 
and grade levels to answer survey items on 
curriculum and instruction is the 
recommended approach, the decision 
regarding what school subjects and grades 
to select is not an easy one. We thus 
recommend a mixture of large and small 
studies on instruction and teaching 
effectiveness within the SASS program. 
When the state of the art is adequate in 
terms of prior instrument development, 
larger numbers of teachers should be 
studied. When the state of the art is less 
adequate, small numbers of teachers 
should be studied in the service of 
instrument development and validation. 

Subject Areas and Grade Levels 

Because so much investment has been 
made in instrument development in math, 
and because the NCTM standards were in 
the vanguard, it seems appropriate to use 
math as one of the target subjects. The 
scope of surveys about math should be 
expanded to include math in the upper 
elementary grades (4-6) along with middle 
school and high school. If costs permit, 
middle and high school science is another 
area in which some useful instruments are 
available. 

In addition to math, Porter (1991) suggests 
English as an important understudied area. 
Leighton, Turnbull, and Mullens (1994) 
suggest history as another possibility. At 
the middle and high school levels, both are 
plausible options, with history having 
somewhat of an edge in terms of existing 
instrumentation. If history was selected as 
a focal subject, the elementary grades 
should again be included. Fifth grade is 
typically the year U.S. history appears in 
the elementary curriculum. So surveys in 
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grades 4-6 would make sense. However, 
the elementary social studies curriculum is 
quite diverse and content items should 
range well beyond history to articulate 
with actual practice. 

Further, within social studies, there is 
contention about the direction the subject 
should take. Both history and social studies 
standards (NCHS, 1994; NCSS, 1995) 
have been formulated and there is 
considerable tension among adherents to 
each set of standards. A SASS survey 
aimed at charting reform in this area, 
would be challenged to accommodate 
differing points of view. 

English/Language Arts standards have 
been released recently (NCTE, 1996) but 
offer little guidance with respect to content 
coverage as they emphasize pedagogy. In 
some cases, English or Language Arts 
consists of instruction primarily geared to 
developing skills in reading and/or writing; 
in other instances the instructional program 
is directed more toward literature. Based 
on all these factors, the choice of 
history/social studies for inclusion in SASS 
might have a slight edge. In any case, we 
envision smaller scale studies in subjects 
beyond math and science oriented 
primarily toward instrument development 
and validation. 

A Caution 

A cautionary note should be sounded with 
regard to the subject-specific focus of this 
discussion. Current instruments and our 
discussion have assumed that instruction is 
compartmentalized by subject. Empirical 
evidence suggests this is still largely true, 
but a number of curricular reforms call for 
more subject matter integration and 



interdisciplinary teaching. It seems 
important to bear this in mind in reviewing 
items for inclusion in SASS and in 
thinking about how to select teachers. 

One hopes that there are teachers whose 
instructional programs are strongly 
integrated for whom answering a more 
conventional survey could be problematic. 
Such teachers may be more often found in 
elementary schools, but high school 
programs emphasizing subject integration 
are also being implemented. 

Instrument Development. Pilot Studies. 
Validity Studies 

Support for survey instrument development 
seems in order as an important step to 
prepare for the next SASS cycle. These 
efforts should be directed at enhancing our 
capability to measure curriculum and 
instruction in subjects hitherto 
understudied-especially elementary math, 
and history/social studies at the 
elementary, middle and high school level. 
Work might also begin on mapping the 
English/language arts curriculum for future 
inclusion in SASS. 

We have already discussed the types of 
items needed to assess the implementation 
of curricular reform. (See section on 
Curricular Reforms.) Item development or 
modification of existing items should also 
go forward in preparation for the next 
cycle of SASS. 

The instrument development projects 
would involve multi-method investigations 
that could determine the validity of pilot 
items and other methods. Effective use of 
teacher logs, collection of teacher 
assignments, exams and other materials; 
textbook analyses, and classroom 
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observations might be incorporated in the 
instrument development process. If new 
item sets are ready for administration in 
SASS, we recommend smaller sample 
studies in the first round so that their 
validity can be established. We agree with 
Burstein et al. (1995) that validation studies 
should regularly accompany the 
introduction of new surveys. Thus 
continuing validity studies should be 
supported during SASS administration. 

Additional small studies conducted through 
SASS (perhaps in Follow-Up surveys) 
might delve into topics of interest to the 
nation from time to time. Illustrative is 
the section of the SASS 1994-95 Teacher 
Followup Questionnaire which inquires 
about portfolio assessments. Inquiry into 
special topics such as this could be a 
regular part of SASS, with only a fraction 
of teacher respondents being asked to 
provide information. In this manner, not 
all teachers would take exactly the same 
set of items, but reliable information could 
still be obtained on a number of interesting 
issues. 
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This paper was written at the request of Policy Study Associates for use by NCES. The author is 
solely responsible for the ideas presented. 

I would like to acknowledge the generous assistance of John Mullens at PSA in providing numerous 
documents and surveys. Andrew Porter and Fred Newmann, both at the University of Wisconsin, 
Madison, Leland Cogan of the TIMSS at Michigan State University, Joan Talbert at CRC at Stanford 
University, and Valerie Lee at University of Michigan, also made surveys or papers available. 

Revision of the paper benefitted from the comments of Dominic Brewer and Cathleen Stasz of the Rand 
Corporation and Pamela Grossman of the University of Washington. 

I have tried to cite sources as appropriate throughout the paper. However, the surveys and papers that 
form the core used in preparing this paper are much like an extended family. Surveys have uncharted 
historical connections to one another, with items borrowed and adapted freely. Many commonalities in 
thinking appear in papers on the topic. I apologize in advance for any omissions in citations or for 
mistakenly citing a source that is not the definitive one. 

Phillip Kaufman in this series suggests linking SASS to a new NELS. 

The CRC survey of California elementary math teachers uses some items from the National Center for 
Research in Teacher Education at Michigan State University. 

Of course the NCTM standards also recommend change in what is taught in math classes. 

Resource use, especially what textbooks and other materials are used, is beyond the scope of this 
paper. The omission does not reflect a lack of importance. 



TOWARD AN ORGANIZATIONAL DATABASE ON 
AMERICA’S SCHOOLS: A PROPOSAL FOR THE FUTURE OF SASS, 
WITH COMMENTS ON SCHOOL REFORM, GOVERNANCE, 

AND FINANCE 1 



David P. Baker, The Catholic University of America and 
American Institutes for Research 



The question before us is what could, and 
should, NCES’s Schools and Staffing 
Survey (SASS) become in the future? In 
other words, what role should subsequent 
collections of SASS play in NCES’s 
statistical program? It is recommended 
here that, while some continuity between 
past and future surveys is important, SASS 
should not just repeat what it has done in 
the last three surveys. 

After three data collections and almost of 
decade of service to NCES, SASS is at a 
cross-roads in its development. One road 
leads on in the same direction set by the 
three completed surveys, with perhaps 
only some minor additions and slight 
modifications to the items. Although I see 
nothing wrong with doing reports similar 
to past ones with fresh data, it is 
nevertheless a limited strategy that will not 
maximize the use of future S ASS’s. To 
essentially repeat what has already been 
done retards emerging developments SASS 
has made over the past three surveys. The 
other road follows these developments 
towards a new SASS that presents NCES 
with an opportunity to provide more 
complex and broader information on 
schools as educational organizations. I, 
therefore, recommend taking this other 
road, one that leads in a new direction. 

The new direction is to make SASS 
NCES’s main vehicle for organizational 
information about the nation’s K-12 



schools. It has, in many ways, already 
become this, but this goal should be made 
more explicit as the survey’s main 
objective. The focus of future SASS’s 
should be broader than in the past by 
including a wider scope of information 
about how American schools are 
organized. It should become the baseline, 
fundamental survey for a host of NCES’s 
efforts in describing elementary and 
secondary schooling in the country. SASS 
has moved in this direction and it should 
move further still beyond a more narrow 
survey on teacher supply and demand to an 
omnibus survey of school organization. 

I recommend a renewed SASS for four 
reasons that are explored in some detail 
below. The reasons involve: (1) what 
SASS has become over the past decade; 

(2) what I perceive as some critical weak 
points in NCES’s statistical program on K- 
12 schooling that need addressing; (3) 
what knowledge and technical capabilities 
about surveying schools have been accrued 
over the last three SASS’s; and (4) what I 
think NCES should collect about how 
schools approach reform, their governance 
structures, and finances in the American 
system of formal education. 

Before discussing the reasons for a new 
SASS, I will first describe what a broader 
organizational survey might look like. 

This description will move through the 
general to the more specific. Then I 



outline one example of how this might be 
done within the current length and scale of 
SASS. This is followed by discussions of 
a new SASS in comparison to the current 
one; SASS and organizational theory; and 
the reasons why a new SASS is useful at 
this point in NCES’s history. 



The New SASS as an 
Organizational Database on 
Schools 

What would an organizational database 
consist of and how different would this 
new SASS be from the last three? What 
basic organizational components should be 
added to the existing organizational 
information collected in the 93-94 SASS? 
What essential information about a school 
should SASS collect for NCES? To 
answer these questions, I see at least four 
main perspectives that an organizational 
survey of schools must take to maximize 
its benefit for NCES. 

1. One Main Focus of the Survey Must Be 
on How a School Organizes Its Main 
Dynamic Components 

Information should be collected on how a 
school organizes its four main components 
of: (1) faculty; (2) student body; (3) 
curriculum/ instruction; and (4) immediate 
outside environment such as parent and 
community groups interested in education. 
By "dynamic" I mean more emphasis on 
how decisions and control move through 
the organization and less emphasis on 
static titles and fixed structures. It is 
better to spend limited questionnaire space 
on, for example, how a departmentalized 
system influences control over teachers 
than on information about the status of 
department heads such as part-time, full- 



time and so forth. The focus on each of 
the four components should be on how 
decisions are made, what decisions have 
recently been made and how they are 
reviewed and changed. The key 
information to capture is organizational 
process not just organizational structure. 
During the planning forum some suggested 
that this kind of information is too 
complex to capture in a survey format. It 
is true that this is not simple information, 
but neither are the types of information 
that SASS currently measures. There is a 
host of detailed research on these aspects 
of school organization that a survey could 
make use of to develop short, but accurate, 
indicators of these components. Examples 
of information needed about each 
component are as follows. 

Teachers. SASS has certainly been a 
major source of information on K-12 
teachers, and a lot of this should be kept 
in a new SASS. But, as I illustrate below, 
some of the teacher information will need 
to be scaled down to make more room for 
information on the other three central 
components of school organization. But 
still we need to know more about how 
teachers and their activities are controlled 
(or not) in schools, who does the 
controlling, and over what issues. We 
also need to know more about what things 
teachers have real decision making power 
over and whether or not it is used. Some 
of this kind of information was collected in 
all three SASS’s, but more direct items 
need to be developed than the questions 
about general influence teachers (and 
others) might have on certain issues. The 
key here is to capture a picture of how 
much administrative control there is over 
teaching and the work conditions of 
teachers and how much decision-making 
power teachers have within the school. 
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For example, are teachers and their 
teaching observed, inspected, and 
assessed? By whom and with what 
purpose? What kinds of collegial control 
is there in the school versus more 
hierarchical control from non-teachers? 
Measures of teacher autonomy and school- 
based management are crucial to collect. 

Students. As the recent NCES Fast 
Response Statistical Survey (FRSS) on 
curricular tracking and student assignment 
to courses shows, NCES’s student-level 
data sets such as HS&B and NELS do not 
tell us many key things about how schools 
organize student (NCES, 1994; Baker, 
Ralph & Manlove, forthcoming). SASS 
could play a very important role in 
collecting information about how schools 
manage their students. A short list of 
examples include information on: 

© Policies about student choice of 

courses at the middle and secondary 
school level 

• Student discipline policies 

• Ways in which the school organizes 
parental involvement in schooling 

• Decisions about programs for 
students and policies about access 
to these programs 

• School policies and management 
philosophies towards student 
abilities 

• Pedagogical approaches to variation 
among students in motivation, 
intelligence and educational and 
occupational goals (i.e., tracking 
and ability grouping) 



There are many such issues to choose from 
and not all can be included, but again the 
key here is to gain a picture of how a 
school makes decisions about students and 
programs for students. Each prior SASS 
has increasingly done some of this, but 
more explicit information on these kinds of 
things should become part of the new 
SASS. 

Curriculum and instruction. A school’s 
curriculum is a central and complex 
component of its educational operation. 

Yet, one of the weakest points within 
NCES’s overall statistical program is 
information on what gets taught and how it 
gets taught. Most of the information on 
what is taught in schools is inferred from 
individual student course taking records or 
from teacher teaching loads; little is 
known about national estimates of school 
organization of the curriculum through 
secondary school courses or coverage time 
in the elementary school. In the past, 
NCES has collected relatively little 
information on curriculum, however there 
have been improvements. The attention 
that the term "Opportunity to Learn" has 
brought to curricular aspects of schooling 
will probably increase the amount of 
information collected on curriculum and 
instruction in the future (Brewer & Stasz, 
1995). A new organizational SASS would 
be a perfect vehicle from which to gather 
basic OTL information on curriculum and 
instruction at the school level. For 
example, from just one item on the fore 
mentioned FRSS on curriculum tracking, 
NCES received for the first time national 
estimates of how public schools organize 
their course structure, requirements and 
enrollments in tenth grade mathematics 
and English. This kind of information 
should be collected for other grades. 
Besides course structure, information 



should be collected about curriculum 
content, such as what is meant by an 
"algebra course" in the eighth grade for 
example. Some parallel information on 
how the curriculum is implemented by 
teachers would be equally useful. Also 
information on how local, state or national 
standards are translated into actual 
curriculum is important to collect, as well 
as information about grading policies and 
how decisions in these areas are made at 
the school level . Some information of this 
type exists in SASS, but it is oriented 
towards representing a teacher’s teaching 
load, not the school’s organization of its 
full curriculum. NCES should know the 
basic dimensions of the K-12 grade 
curriculum in the nation’s schools and a 
new SASS is a good way to do this. 
Because of the complexity of curriculum, a 
new SASS should not attempt to do an in- 
depth collection on this area of schooling, 
but some basic information would go a 
long way toward improving NCES’s 
reporting on curricular and instructional 
information. 

Immediate outside environment. Schools are 
linked to the community they serve 
through a series of semi-formal 
organizations such as parent organizations, 
business councils and other community 
groups. Schools interact with these and 
have varying roles in their creation and 
influence on education in the local 
community. Although these kinds of 
organizations may not be as fundamental 
to the day-to-day operation of a school as 
teachers, students and curriculum, they do 
play a significant role in school 
organization. A new organizational SASS 
should learn more about how schools 
connect with these kinds of organizations. 



Other components . Certainly there are 
other parts to a school that make up its 
total organizational picture. The most 
obvious one without its own section above 
is the school’s administration. In part this 
is because the administration plays a role 
in organizing and managing each of the 
four main components of the school, so it 
is included in this way. Additionally, 
other information about the structure and 
function of the administration would be 
essential to complete the organizational 
description of a school. There may be 
other organizational components that a new 
SASS could include, but I have tried to 
provide the essential minimum. 

2. A Second Main Focus of the Survey 
Should Be on How Much of the School’s 
Organization is Influenced bv District. State 
and Federal Educational Agencies 

One of the unique features of the 
American educational system is its local 
administration of schools. From the 
beginning of American formal education, 
communities organized and controlled their 
own schools. This arrangement carried 
over into the period during which an 
extensive public school system was 
constructed by local governments. The 
American public school then is a local 
entity. However, over the past century 
different levels of government have taken a 
larger role in the regulation and control of 
schooling. Local education agencies 
(LEAs) are influenced by other 
government agencies at the state and 
federal levels. Added to this is the 
variation across the country as to which 
different levels of government have control 
over which specific areas of schooling. 
There is also a sizable private school 
sector at both the elementary and 
secondary levels which, compared to 
private schooling in many other developed 
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nations, is relatively un- regulated by 
government. 

Taken together these features make the 
American school system local, varied 
across place, and multi-governed. And 
this has direct and important consequences. 
For example, it may be far more difficult 
to standardize education within the U.S. 
than elsewhere. This feature is often put 
forth as one reason why the width of the 
distribution of academic performance 
among American students is large 
compared to that found among students in 
other countries. Also it is not necessarily 
easy to determine what is occurring within 
schools in this kind of a system. Indeed 
one of the reasons for why NCES gathers 
most of its information through surveys 
instead of central administrative records, 
as is commonly done in more centralized 
education systems of other countries, is 
because the unique governance structure of 
American schooling. 

The suspicion, at least, is that there could 
be large variation in schooling across the 
nation. Some assume this and hold it as a 
positive in the form of high potential for 
innovation in this kind of a system; hence, 
for example, the reform notion of further 
decentralization of an already relatively 
decentralized system. Others present this 
more negatively and suggest that 
parochialism and incompetence in school 
administration can often go unnoticed and 
unchanged in this kind of a system. 
Regardless of which view is more 
accurate, this unique governance structure 
is a fundamental feature of American 
schooling and NCES needs to provide as 
much information on this feature’s impact 
on schooling nation-wide as is possible. 



It is important, then, for a survey of 
school organization to be able to gain 
some information on the influence that a 
multi-governance structure has on schools 
and what they do on day-to-day basis. 
Currently SASS includes some questions 
about influence from the district in some 
broad areas, but as stated above, these are 
not very specific and do not help to really 
distinguish between different kinds of 
governance environments found around the 
country and across public and private 
sectors. The new SASS should include 
information about how much influence 
various levels of educational governance 
have on the way a school organizes the 
four main components described above. 

This is particularly true of the district for 
public schools and other similar supra- 
administration of some private schools 
such as the diocese for the Catholic 
schools. M. Ross’ paper in this collection 
focuses on school districts. Many of the 
managerial issues that I outline above have 
their origins in district offices even though 
implementation and control are at the 
building level. Some information on 
policy setting and school control by the 
district is essential to complete the picture 
of the organizational nature of a school. 
But even having said this, I still 
recommend that the primary unit of survey 
focus be the school for the reasons that I 
list below. 

Although this is a complicated area, NCES 
should at least know some of the basic 
levels of influence that district, state, and 
federal agencies have on schools. And 
this needs to be more specific than the 
general impression of the principal in 
terms of global influence, as is now asked 
in SASS. For example, which kinds of 
schools are bound by district rules and 



procedures for the hiring of teachers, 
assessments of instruction, creation of 
student programs, curricular issues and so 
forth? How much state control and 
regulation reaches the school in these 
areas? How much federal regulation 
reaches the school and what form does it 
take? 

3. SASS Must Gather More Information on 
Financial Resources and Their Flows to the 
Point of Instruction 

Resources available to schools, broadly 
defined, will increasingly be a topic of 
policy debate as funds for education meet 
with stiffer competition for other activities. 
Although a new SASS can not and should 
not replicate the detailed school finance 
work already being done by NCES, it 
should include enough information on 
costs and resources to make some 
estimates of how schools manage 
resources. The current information on 
teacher salary schedules is important to 
retain since it helps to generate cost 
estimates. 2 

Since for the public sector at least, most 
finance and its controls are set beyond the 
school, both NCES’s F33 at the local 
district and LEA level and the NPEF 
survey at the state level capture basic 
national financial information. What is 
missing is how much financial resources 
are transmitted into school level resources 
and how this is done. How much of the 
overall resources flow to the point of 
instruction? Besides teacher salaries and 
quality, one additional way SASS could 
add organizational information on this is to 
examine what instructional resources 
teachers have access to within sampled 
schools. For example, in more affluent 
districts, teachers can be assisted by an 



array of other instruction personnel, such 
as curriculum specialist, resource teachers 
and other classroom level specialists, 
which adds to the total instructional 
resources reaching the students. Optimally 
NCES should know what portion of 
students and teachers have access to these 
resources and how many of these actually 
make use of them. Additionally it would 
be useful to know the degree to which 
resources are devoted to developing 
curriculum and instruction in the school 
and to parent and community organizations 
connected to the school. Even information 
on very basic instructional resources, such 
as supplies and simple technologies, could 
be useful information about overall 
resources and school organization. 

4. A New SASS Must Include Measures of 
School-level Educational Outcomes 

The utility of what is proposed here for a 
new organizational SASS would be greatly 
compromised if it did not somehow 
include information on educational 
outcomes. Even though the details of this 
are explored elsewhere in this planning 
process, I wanted to make my separate 
recommendation known. For numerous 
reasons, NCES needs to incorporate into 
SASS school-level information on student 
achievement, promotion, dropout, 
disciplinary actions, college application 
and so forth. Currently SASS does a 
small amount of this in characterizing a 
school’s student body, but more should be 
done in the next SASS. The central issue 
that reoccurs within the American 
educational policy arena is what outcomes 
are associated with what organizational 
features. From macro "school effects" 
research to work on the micro 
improvement of instruction, the 
assumption is made that schools and 



outcomes of students are linked. Some 
take a more explicit organizational view of 
this assumption, others are more skeptical, 
but nevertheless this idea is the backbone 
of most education policy. So for NCES to 
maximize its impact on the public 
discourse on education, it should maximize 
its ability to provide information related to 
this most basic of policy assumptions. 
Certainly both the NAEP and 
HS&B/NELS surveys yield important 
outcome information for NCES, but 
neither has the perspective that a new 
SASS could provide. NAEP was not 
originally designed to examine any inputs 
to achievement beyond technical controls 
for assessment estimates. And the 
HS&B/NELS surveys contain some school 
organizational data and student outcomes, 
neither is as extensive as what is proposed 
here. Also, the main focus of these 
longitudinal surveys on the student’s 
progression through school and into the 
labor market shapes the survey to 
answering more questions about what 
influences individual achievement and 
attainment than what types of school 
organization yield what types of outcomes 
for their students. 

This is perhaps a fine distinction, but one 
that I think is worth considering. We 
actually do not know much about how 
various differences in school organization 
influence student outcomes in the 
aggregate. This was, for example, the 
original intent of the first Coleman report 
(1966), but the study used what would 
now be considered a very simplistic notion 
of school resources and almost completely 
ignored the organizational dimension of 
schools. Thus when family and student 
background effects were found to "swamp" 
most school level effects, this line of 
research went in another direction. With 



an omnibus, organizational SASS that 
included educational outcomes, NCES 
could make a major contribution to the 
debate about what kinds of schools 
produce what kinds of outcomes and 
perhaps add insight on how this comes 
about. 

I am not necessarily arguing for a full 
assessment of students in each surveyed 
school in SASS. Perhaps some sub- 
sampling could be done; or some form of 
linking with NAEP; or even some use of 
extant student assessments within schools 
could be made. But the point is that for 
SASS to become a central vehicle for 
NCES, along with NAEP and NELS, it 
must include information on school 
outcomes. 

How Could a New Organizational 
SASS Be Implemented? 

What is proposed below is just one 
example, a sketch really, of what might be 
done to incorporate the ideas above to 
make SASS an omnibus, organizational 
survey of K-12 schools. In doing this I 
make several assumptions. 

First, I assume that the next round of 
SASS will have to be designed within 
roughly the same length and response 
burden parameters as was in the 1993-94 
survey. This makes planning additions 
and changes more or less a zero sum 
operation. For something to be added, 
something of equal size needs to be 
deleted. It also makes the job of designing 
a new approach of SASS difficult. This is 
particularly so for me since I find many of 
the current SASS items useful and I do not 
easily part with them. 
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Second, I assume that the school will be 
the main unit of focus of a new 
organizational SASS. Information from 
supra-units such as districts and other 
LEAs (private or public) would be mostly 
focused on the sampled school. The same 
would be true for teachers in that the focus 
would be on their experience with this 
school, as is currently the case with some 
parts of the teacher questionnaire. The 
logic here is the same as in HS&B or 
NELS with one primary unit of analysis 
with supporting information from others, 
but unlike these student level surveys, the 
focus of a new SASS is explicitly on the 
school . 3 

Third, I assume that most of the basic 
questions about schools such as 
enrollments, location, school types and so 
forth (i.e., many of things collected in 
section A of the School Questionnaire) 
would remain. Although the SASS staff 
might want to consider more efficient and 
less burdensome ways to generate this 
from principals. For example, would it be 
more efficient if principals were given a 
listing of their school’s basic information 
from CCD that were used to create the 
sample and be asked to verify the accuracy 
of these? 

Fourth, let us assume that approximately 
four to five items per revised questionnaire 
about each of the four components 
described above would provide enough 
information to draw an effective picture of 
the organization of the school. 

Fifth, I assume that what is proposed here 
can be adapted for schools in the private 
sector and Indian schools. Also, there will 
need to be some specific adaptations for 
elementary and secondary schools. 



Finally, for this exercise, I assume that the 
libraries component stays as a separate 
component and the Teacher Listing Form 
stays as a way to generate any teacher 
sample. The question then is what might 
be done with the Teacher Demand and 
Shortage/District, Principal, School, and 
School Teacher questionnaires to make 
room from items that capture the ideas 
above? 

Teacher Demand and Shortage/District 
Questionnaire 

The Teacher Demand/District 
questionnaire should be less oriented 
towards general district programs and 
more towards whether or not the district 
sets policy and procedures on issues that 
correspond to the management of teachers, 
students, curriculum, and other parts of 
the school. These items should be geared 
toward the district’s school in the SASS 
sample, not just in general terms about all 
schools in the district. Some new items 
will need to be developed. For new items 
and for many items currently on this 
questionnaire what needs to be added is 
some clear indication of the relative 
control the district (or appropriate LEA) 
has over the target school vis-a-vis 
selected areas of school operations. The 
items should indicate where the 
relationship between the district and a 
school is on a range of control from one of 
"district set enforced policy" to "policy as 
a guideline with considerable school-level 
discretion." Also, this should be the place 
where some information is gathered about 
the influence of state policies and federal 
policy impact. 

Specifically, I would recommend that 
section A, B, and C of this questionnaire 
remain basically the same with the addition 
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of some indication in section B about the 
control of teacher hiring policies at the 
school level by the district or LEA. 

Sections D and E should be changed the 
most. Section D gathers information on 
several federal programs and one local 
program of "choice." This section is a 
mini-survey of districts since none of this 
is tied to the target school. Also, some of 
these same federal programs are asked 
about in the school questionnaire. Does 
NCES need both estimates? Section D 
needs to be redone; it should provide a 
picture of how district, state, and federal 
programs have an impact on the target 
school. As it cannot necessarily include 
all such programs, a sampling is enough, 
but the key is to capture something about 
if and how federal, state, and district 
programs have an impact on the target 
school. Section E should be the place 
where district policies and governance 
about each of the main school components 
are examined as suggested above. I 
recommend that what is currently in this 
section E be deleted to make room for new 
items. 

Principal and School Questionnaires 

Most of what I have proposed about a 
more organizationally orientated SASS 
falls on these two questionnaires. The 
current Principal Survey asks a number of 
questions about the principal’s background. 
I would like to see the general area kept, 
but greatly reduced. Items 1 through 23 
and 26 through 30 should be cut down by 
at least two-thirds. I think just some of the 
basics about principal background is 
sufficient. In their place new items should 
be developed that examine how the 
administration of this school manages 
faculty, students, curriculum, and outside 
influence on the school. The decision- 



making items 24 and 25 should be made 
more specific along the focus described 
above; right now they are far too general 
to be of much help. If there is space left 
after this, here is the place where some 
information on the principal’s management 
philosophies and approaches should be 
collected. 

The School Questionnaire has four 
sections. Section A on school 
characteristics should be kept, but perhaps 
it could be collected in a less burdensome 
fashion (see CCD suggestion above). 
Section B is the basic staffing of the 
school, which is important to keep, even 
though it does not tell us much about the 
direct management of the school’s faculty 
and staff. Section C is on programs and 
services. I would suggest that this section 
be redone and be the section where most 
of the way the school organizes students 
and their programs is collected. Some of 
the programs here are the same ones asked 
at the district level and if the district items 
are removed, these will have to stay. My 
problem here is that most of the programs 
here are entitlement types for special 
student populations which may be a small 
part of what the "average school" does 
with most of its students. This is the place 
where some of more extensive information 
on the curriculum and student flows across 
courses should be collected. Section D is 
a mixture of items. Item 33a is important 
and is the kind that I recommend in 
general on decision-making. I would give 
up the other items in the section in favor 
of a more systematic view of how the 
school organizes its main components. 

School Teacher Questionnaire 

This section of SASS is the hardest to 
change, but fortunately not much needs 
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change to bring it into line with what I 
have proposed. The information collected 
here provides much of the data for the 
national profiles of teaching as a 
profession from a number of perspectives. 
It has proved very important to NCES as a 
way to monitor the nation’s K-12 faculty 
in a way not available from any of its 
other surveys. There are nine sections to 
this questionnaire; I would suggest keeping 
most of the items in all sections. Sections 
A, B, G, and H provide the basic 
dimensions of the teacher and her/his job. 
Section I is a small single item on LEP 
and section J is a technical item. Section 
C is a long section on training. Much of 
this captures information on instructional 
support for the teacher from the target 
school, so it’s very useful. Section D is 
on teaching load. If a new SASS included 
more school level questions on course 
structure, some of this might replace the 
teacher level items in this section. Section 
E is very important and most of the items 
are exactly in line with how teachers 
perceive and experience the school as an 
organization. While one could quibble 
with some of the items in this section, in 
the main, it is very useful to an 
organizational view of the school from the 
teacher’s point of view. 

This example of a way to implement a new 
organizational SASS does not mean an 
extensive overhaul of the full survey. The 
Teacher/Demand District questionnaire is 
the most changed followed by the Principal 
and School questionnaires. Most of the 
School Teacher questionnaire should 
remain the same with some change in 
emphasis of some items. Without 
constructing the actual new items and re- 
working current items it is hard to tell how 
naive this implementation plan is, but it is 
an approximation of what would have to 



be done to a new SASS. This plan does 
not include what needs to be done to SASS 
to incorporate more school level outcomes 
such as achievement, promotion, and so 
forth (see point IV above). 

How Different in Focus Would a 
New Organizational SASS Be from 
the Current SASS? 

SASS has evolved to have three main foci: 
(1) teacher demand and shortage; (2) 
condition of teachers and teaching as a 
profession; (3) basic organizational 
structure of schools. And the importance 
of the foci in terms of initial questionnaire 
construction corresponds to the above 
order. What I am suggesting here is a 
reversal of that order. Make the 
organizational focus more prominent with 
a correspondingly larger share of space on 
certain questionnaires; keep the focus on 
teaching and its profession as the second 
focus and move the issue of teacher 
demand and shortage to be the third focus. 

I suggest lowering the emphasis of SASS 
on teacher demand and shortage for 
several reasons. Although I think that the 
general area of supply and demand is 
useful and there has been some valuable 
work done on this for NCES, it might be 
that too much of SASS is used for this 
purpose. From past work we now know 
how to estimate basic supply and demand 
models efficiently from relatively few 
variables, the real question is how detailed 
and nuanced do we need to make the 
estimates by adding further variables (Boe 
& Gilford, 1992). Given that the issue of 
supply and demand for teachers has not 
proved to be the large policy issue that it 
was once thought to be, perhaps NCES 
should only provide the most basic 
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estimate of teacher supply and demand. 
For example, does NCES currently need 
estimates of teacher supply and demand 
from both a district and school 
perspective. I understand that aggregated 
estimates are built up from the unit below 
(Barro, 1992), but if we have good state 
estimates constructed from district data 
would not that be enough to provide 
information about the general issue? And 
if this basic data indicated a large problem 
or other related issues arose, perhaps 
supplemental surveys could be used for a 
more detailed assessment. 



Organizational Theory and the 
New SASS 

Perhaps the most useful of NCES surveys 
are those that correspond to strong 
research literatures and theories which in 
turn have a major impact on educational 
research and policy. This kind of a 
connection links a NCES survey to a 
research field and related policy domains. 
This is helpful in both the development of 
the survey and in maximizing the influence 
the gathered information has on the 
education establishment. The link between 
assessment and psychometric theory and 
NAEP, and the link between theory on 
both adolescent development and 
educational attainment and HS&B/NELS 
are two examples of NCES surveys that 
are strengthened by this kind of a 
connection. An advantage of shifting the 
focus of SASS to a broader organizational 
one is the ability to have SASS correspond 
to the large literature on organizational 
theory. 

The proposed approach is in line with 
what research and theory on organizations 
in general, as well as on schools as 



organizations in specific, would suggest 
about planning an organizational database 
on schools. Needless to say, the literature 
on formal, complex organizations and the 
parallel one on schools as formal, complex 
organizations is too voluminous to review 
here. But there are several important 
lessons from this literature that are helpful 
in designing a new organizational SASS. 4 

Although Formal Organizations Have Far 
More "Sloppiness" in Their Boundaries 
Than Originally Assumed, They Are 
Observable and Can Be Studied as 
Organizations 

Where an organization stops and starts is 
not as easily determined as what was once 
thought. Research on all kinds of 
organizations in both the public and private 
sectors shows that formal organizations 
have a certain fuzziness to their 
boundaries. A school may be a school, 
but it is also part of a district or a set of 
private schools. Or a PTA is a part of a 
school but it is also an organization within 
a school that also has connections outside 
of the school and so forth. Nevertheless 
organizational research has coped with 
these characteristics of formal 
organizations and has shown that, by in 
large, organizations such as schools can be 
assumed to have enough of a traceable 
boundary to make them a suitable entity 
for study. So it is possible to consider 
organizations such as schools as discrete 
units that can be the focus of a survey. 

Past SASS’s have already shown this to be 
operational. The new SASS proposed here 
would increase the focus on the school as 
the main unit of the survey with, as is now 
the case, supporting data collected from 
teachers and districts or LEAs. 
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Organizational Actors Can Provide Reliable 
and Accurate Assessments of Collective 
Processes that Make Up an Organization 

This is a basic tenet of organizational 
research. Surveying people holding an 
organizational role is often used as a 
method of collecting information about 
organizations. There are, of course, some 
problems with having individual actors 
represent whole organizations, but these 
problems turn out to be no greater and not 
very dissimilar from problems with 
surveying individuals about themselves. 
Additionally, surveying actors about 
organizations has the advantage of 
providing multiple views through multiple 
actors, as has been used in SASS reports 
on schools and teaching conditions derived 
from aggregated information from teachers 
at the same school (e.g., Ingersoll & 
Bobbitt, 1995). Organizations offer the 
additional advantage of generating other 
useable sources of information about 
themselves such as administrative records 
and financial records (e.g., Scheuren, 
1995). 

Technical Processes within Organizations 
That Connect an Organization’s Goals to Its 
Output Are Far Less Tightly Coupled Than 
Was Once Assumed. Making Information on 
Social Organization More Informative Than 
Information on Organizational Structure 

The way many organizations work, or how 
they move from goals to means to ends, is 
best characterized by an image of "loosely 
coupled" connections (Weick, 1979; Orton 
& Weick, 1990). This is particularly true 
of schools. The outcomes of achievement 
and socialization of students are not easily 
connected to many of the processes within 
the school. In other words, a mechanical 
view of schools as organizations is too 
simplistic to understand how schools really 
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work. Observing the process of 
management and the social organization of 
schools instead of just the structure of 
schools leads to a more accurate 
understanding of schools (Barr & Dreeben, 
1983; Weick, 1982). Expanding the 
organizational scope of SASS to examine 
organizational processes is consistent with 
the widely accepted "loosely coupled" 
perspective on the way organizations 
work. 

Organizations Are Less Rational Than They 
Were Once Thought to Be 

This central observation about 
organizations suggests that they are not 
infinitely rational, rather organizations use 
a form of "bounded rationality" (Simon, 
1955). This idea is related to point 3 
directly above, but adds to it the notion 
that decision-making in organizations is the 
key operation to observe to determine their 
basic essence (March & Olsen, 1976). 

And further, decision-making is also not 
mechanical but a very compromised 
process. The point then for SASS is to 
collect information on how the workings of 
an administrative system in a school makes 
decisions, exerts control, and how that 
same administration can ignore other areas 
(Hannaway, 1989). This kind of 
information will yield a more accurate 
description of schools for policy-makers 
interested in school reform. 

Why a Database on Schools as 
Organizations? 

As mentioned, there are four main reasons 
why I recommend that SASS be renewed. 
Let me briefly describe each one. 
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Reason 1: What SASS Has Become 

A recent NCES working paper noted that 
"SASS is an unusual education survey" 
(Ingersoll, 1995). Meaning perhaps, that 
the reason behind the original design of 
SASS and its intended statistical 
contribution are very different from other 
NCES surveys. SASS was not designed to 
measure the academic progress of K-12 
students, nor was it designed to measure 
the impact of schools on academic 
learning, nor was it designed to weigh the 
costs and benefits of federal educational 
entitlement programs. SASS was 
originally designed to provide information 
on K-12 teachers, with special emphasis on 
teacher supply, demand, and quality. But 
in the course of doing that, SASS has also 
provided information on schools. It is the 
only NCES data set in which large 
portions of the survey were designed 
around the school as a unit of analysis. 

The teacher sample is representative of 
teachers, but even here a significant 
portion of the information gathered refers 
to the teacher’s school. Without maybe 
initially intending to do so, SASS has 
evolved in the direction proposed here. 

Reason 2: A New SASS Strengthens 
NCES’s Overall Statistical Program 

When I consider the full array of NCES’s 
K-12 statistical program, I am always 
struck by several weaknesses among what 
is otherwise a strong program. In general, 

I think that too much of the K-12 program 
is focused on student level information. In 
part this was historically determined by 
earlier concern over the effects of federal 
programs on individual students, such as in 
the original motivation behind the design 
of HS&B to assess federal programs and 
college attendance. But it is also a 
function of the importance of measuring 



student achievement and the role that the 
Department of Education has come to play 
in education. The federal role in funding 
programs for special populations and 
undertaking summary evaluations of 
national achievement give NCES a 
powerful motivation for NCES to examine 
what students learn and how are they 
progressing in school. To some degree, 
what has been pushed out of the way by 
this strong agenda is information on how 
schools are organized. 

The lack of a survey chiefly dedicated to 
schools as organizations, instead of the 
schooling of students or the achievement 
of students, is a weak point that leads to 
some peculiar holes in NCES’s reporting. 
For example, as I mentioned above, NCES 
should be able to report much more about 
the K-12 curriculum and how it is 
implemented in schools. It is a salient gap 
when NCES can not provide more 
information on such a central component 
of schooling. A related example is that 
NCES lacks information on how schools 
react to various policy changes and 
reforms over time. Federal policymakers 
are often interested in precisely this 
question, and increasingly state educational 
agencies are too. Without a survey 
dedicated to schools, this becomes very 
difficult to do. (I take up the issue of 
school reform and a new SASS below.) 

Rpasnn 3: SASS Has Accrued an Impressive 
Set of Technical Capabilities about 
Surveying Schools 

Faced with the task of developing 
nationally representative samples of the 
wide variety of K-12 schools in the 
country, the SASS project over the past 
decade has generated significant and 
sophisticated techniques in surveying 
schools. Representative surveying of any 



population of organizations can be difficult 
given the complexities of population 
dynamics of organizations (e.g., Hannan & 
Freeman, 1989). Births and deaths of 
organizations can be rapid, transformations 
of organizational boundaries often occur, 
and these, plus other similar phenomena, 
are not necessarily spread randomly 
throughout a population of organizations- 
in short a survey design nightmare. A 
survey of schools faces the same problems. 
For example, significant private sector of 
schooling in the United States, which 
accounts for about one-fourth of all 
secondary schools, offers a particular 
challenge to a survey. Similarly, the local 
administration of public schools in terms 
of mergers, births, and deaths makes 
surveying organizations just within the 
public sector difficult. In addition to 
sampling challenges, developing reliable 
and valid questionnaires for actors within 
organizations is also a major undertaking. 
As is evident from substantive SASS 
reports and the project’s own technical 
assessments, the SASS team has the ability 
to generate representative samples of 
schools and collect useful organizational 
information beyond a survey of teachers or 
staffing needs. This is a major capability 
that should be enhanced further. It has the 
potential for NCES equal to that of the 
assessment capabilities accrued from 
NAEP. NCES should recognize this and 
build upon it. 

Reason 4: A New SASS Would Be an 
Important Vehicle to Add to NCES’s 
Capability to Provide Information on School 
Governance. Finance, and Reform 

Because these three areas are the focus of 
much discussion throughout the American 
education establishment, they will remain 
as important topics for NCES for some 
time. Most of what I have already 



proposed is directly related to school 
governance, which is essentially another 
term for school management (Raywid, 
1991), so not much more needs to be said 
about this as a reason to undertake a new 
organizational SASS. Similarly the 
importance of SASS’s contribution to 
reporting how financial resources are 
turned into classroom resources has been 
described above (section 3 of study 
perspectives). So far I have said little 
about school reform and a new SASS even 
though it is a prime example of why an 
organizational SASS should be done. This 
is explored in the next section. 



A New SASS and Information 
about School Reform 

Even though school reform is central to 
the SASS planning process, I have waited 
to discuss it until after establishing the idea 
of an omnibus organizational survey of 
schools that does not necessarily focus on 
any particular reform or restructuring 
trend. I did this for two reasons; first, 
because of what I see as the nature of 
reform in the American system and 
second, because of what I will recommend 
NCES do to capture relevant reform 
information. 

There is a paradox about school reform in 
the United States: the country provides a 
large amount of school reform movements 
but the content of these reforms is mostly 
cyclical. Over the past century there has 
at any one point in time been ample, 
sustained and serious interest in reform 
schools. And these reforms have had 
consequences; the development of the age- 
graded school, the Carnegie unit and the 
core academic curriculum, school racial 
desegregation, and ability grouping are 
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some examples of defining educational 
reforms (e.g., Mirel, 1994). But while 
there is the image when the system needs 
fixing a suitable reform is fashioned, it is 
rarely this simple. Reforms at any point 
in time represent political positions about 
schools that can extend beyond to larger 
political orientations. Shifts in the content 
of reform are shifts in political power. 
Deciding the content of educational reform 
often pits a host of local concerns, 
resources and political orientations against 
often competing educational professions 
and loosely linked national business and 
national political parties (Manlove & 
Baker, 1994). The ups and downs of 
political movements have much to do with 
what is important educational reform at 
any point in time. 

This makes it difficult for a statistical 
agency like NCES to plan long range 
surveys about particular school reforms. 
Although reform in general is a constant 
topic of deep interest within the American 
education establishment, it is difficult to 
decide what exactly should be addressed 
within an ongoing statistics program. 

What is "hot" today is not tomorrow, but 
it may be back in ten years. If a project 
like SASS, which plans to be a long term 
series of surveys, is initially too oriented 
toward a particular trend in education it 
runs the risk of eventually becoming 
irrelevant with the rise and fall of specific 
reform issues. And irrelevancy is the 
worse of all possible fates for any 
statistical program. Therefore I would 
recommend that no matter how 
fundamental, how trendy, how earth- 
shattering any single issue seems for the 
country’s educational establishment at any 
point in time, NCES should not establish 
major, long-term surveys around such 
issues. Rather the key is to think of ways 



to capture information about reform 
without being tied to any one particular 
trend over a lengthy time. 

An omnibus survey proposed here offers a 
flexible way to collect information about 
reform without falling into the trap of 
reform cycles. This kind of a SASS offers 
a reasonable organizational baseline from 
which to undertake supplemental surveys 
of particular reform issues aimed at 
schools. In the year or so after the main 
data collection, a Fast Response Survey 
System (FRSS) or similar means to collect 
information on a sub-sample of SASS 
schools could be used for specific reform 
issues. Then this information can be 
merged with the larger organizational 
database for a powerful set of information. 
The curricular tracking FRSS, using this 
design, has shown that this is technically 
feasible, reasonable efficient, tolerable on 
school response burdens, and analytically 
profitable. A variation on this idea would 
be separate modules of questions about 
specific reform issues given to sub-samples 
within the overall SASS sample during the 
major data collection. Maintaining a basic 
survey of school organization at regular 
intervals with the option to combine this 
smaller, focused data collections on 
passing issues of concern to the American 
education establishment is an efficient and 
flexible way for NCES to stay current. 

Finally, it should be pointed out that the 
current school reform movement happens 
to be heavily oriented towards some of the 
school management issues that I 
recommended a new SASS collect (e.g., 
Elmore, 1992). This, in the short term, is 
one additional advantage to what is 
proposed here. But given what the history 
of educational reform movements in the 
United States shows, I would not predict 



this particular focus will last. The content 
and focus of reform will surely change, 
but a flexible combination of a omnibus 
survey of school organization and other 
supplemental collections will continue to 
be the best way for NCES to provide a 
wide range of information about the 
nation’s elementary and secondary schools 
and any reform issues. 
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As is evident from the title, I have taken some liberty with my assignment to prepare a statement on 
what the next Schools and Staffing Survey (SASS) should include in terms of school reform, 
governance and finances. Although I address each of these, I do so by proposing that the next SASS 
be made into an omnibus survey about the internal organization of K-12 schools. The main focus of 
SASS should be on how schools manage faculty, students, curriculum, and resources; how decisions 
are made about these; and what kinds of administrative controls are in place to implement decisions. 
The objective is to collect information that would provide NCES with a basic picture of how schools 
work as organizations. Then, if need be, supplemental information could be added to address specific 
trends in education as they occur. A re-design of SASS presents NCES with a major opportunity to 
capture school-level information that will increasingly be of importance to its statistical program. 

I propose reversing the original order of priorities of SASS to one of first, organizational and 
managerial information, then second, teacher and teaching information, and lastly, teacher supply and 
shortages. The past SASS s steadily collected more school organization and managerial information 
over the course of the last three surveys, so while what I propose would be a renewed SASS, it would 
not be a radical shift for the SASS project. Additionally I recommend that much of the current SASS 
remain the same, but with some key modifications to crucial questionnaires. As instructed, I have not 
written actual new items, but I have tried to describe what new items should be constructed, where they 
should be placed in the survey and what their informational intent must be. 

Also, as instructed, I have written this for the "insider" reader who is familiar with NCES and SASS. 
For those readers who are not "insiders," two publications make good companions to this document: 
the most recent Programs and Plans of the National Center for Education Statistics and SASS and PSS 
Questionnaires, 1993-1994. 

Much of what is written here originates from an internal memorandum Tom Smith, Nabeel Alsalam, 
and I prepared while I was an AERA Senior Fellow at NCES (Baker et al., 1994). Although I assume 
complete responsibility for any still-half-baked ideas, I owe my two colleagues much for their help in 
thinking about an organizational survey of schools for NCES. I also would like to thank Joel Sherman, 
Tom Parish, and Jay Chambers at AIR for their helpful comments about SASS and finance issues. And 
thanks to Maryellen Schaub for her comments on an early draft. Lastly, thanks goes to John Mullens 
at PSA for his kindly monitoring of my progress during work on this document. 

I have made a few changes to this draft after the January 25, 1996, presentation to NCES. I would 
like to thank Susan Fuhrman for her helpful comments. The revisions here were made in light of the 
comments in other papers made at the half-dozen sessions that I attended and from my reading of all 
the other planning papers. 

The ideas and recommendations herein are those solely of the author acting as a private consultant to 
NCES and do not necessarily represent the positions of any organization with which the author is 
affiliated. 

SASS could prove a vehicle to fill the gap on private school finance, although this is a separate issue 
from what is recommended here (see Garrett, 1996). 

Some might ask, why not focus on classrooms? Classrooms are the key point of instruction and are 
arguably the most basic organizational unit in the formal education process. This is true, but equally 
true is that inputs made to classrooms and decisions about what occurs there are made at the school- 
level (or above). Schools are organizations controlling sets of classrooms. Some information from a 
subset of a school’s classrooms might be useful, but the main focus of SASS should be kept on the 
school as the primary organizational unit of survey. 



4 . 



Just a few citations are offered in the following sections as examples on the larger literature on each 
point; they do not represent a full review. 
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TECHNOLOGY FOR K-12 EDUCATION: 
ASKING THE RIGHT QUESTIONS 

Kathleen Fulton, University of Maryland 



Overview 

Since the introduction of microcomputers 
into schools in the early 1980s, there has 
been widespread fascination with the role 
technology can play in education. The 
promise of computers 1 raises a wide range 
of policy questions: Do they improve 
education? Are they worth the cost? Are 
they being used appropriately? 

To answer these and other questions, 
policy makers and educators need solid 
data about how technology is used in the 
classroom, how it affects teaching and 
learning, and the benefits it provides to 
students and to teachers. However, much 
of the available data to date has focused on 
the number of computers and related 
equipment in schools. While an important 
piece of the puzzle, this data has limited 
value for answering most policy questions 
because it gives little insight as to how 
computers are actually being used by 
teachers and students. Similarly, surveys 
of state policies regarding technology are 
not clear predictors of how much, how 
often, and in what ways various 
technologies are used in K-12 classrooms. 

As we conceptualize the data that would 
adequately address key pedagogical and 
policy concerns regarding educational 
technology, we should consider several 
important points: 

(1) The location of computer 

equipment and other technologies 



within schools affects its usage and 
impact on teaching and learning. 

To assess this impact, we need to 
know not only what equipment is 
available, but also how and where 
it is available, to whom it is 
available and how easily it can be 
accessed. 

(2) Effective adoption of computers in 
schools is not a one-time event; it 
requires ongoing exploration and 
experimentation by teachers and 
students alike. Teacher training is 
critical to the effective use of 
classroom technologies, but the 
kind of training computer 
technologies demand must be 
accompanied by continuing 
just-in-time support as needed. 
Without considering the stages of 
adoption of new technologies, and 
how that adoption is supported, 
data about computer usage will 
continue to be very difficult to 
interpret. 

(3) How computers are used in schools 
is heavily influenced by teachers’ 
conceptions and past experiences. 
For teachers to use technology, 
they must see its value for teaching 
and learning in ways that make 
sense to them. Because the 
definition of what constitutes "best" 
use of technology continues to 
evolve and change, the process of 
technology adoption has been 
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further complicated. Educators 
need opportunities to develop 
visions of how technologies can be 
used to support those aspects of 
teaching and learning that are 
central to them in their classrooms 
today, and opportunities to grow in 
facility and comfort with 
technologies as they evolve. 

(4) Computers and other technologies 
can be adapted to traditional models 
of education, or they can support 
entirely new ways of teaching and 
learning. To formulate and 
interpret data on the impacts of 
educational technologies, we must 
consider the instructional models 
that underlie their design, 
implementation, and use. If the use 
supports new approaches to 
instruction, it is appropriate to 
consider measures and outcomes 
that go beyond traditional student 
scores on standardized tests. In 
addition to alternative measures of 
student learning, teacher 
productivity, empowerment, and 
professional growth are also 
important factors that should be 
better understood and appreciated. 

These issues raise important implications 
for NCES in its mission to collect 
information of value to policymakers. If 
educators are to understand the role of 
technology in schools, they need better 
ways to collect, compare, and evaluate 
data that address these issues. While this 
kind of data is not the sort commonly 
collected in the School and Staffing 
Survey, it may be appropriate for NCES to 
consider supporting other forms of data 
collection, including in-depth studies, 



cross-sectional and longitudinal surveys, 
and combinations of survey and targeted 
comparative case studies, to help provide 
important and currently unavailable 
information needed by educational 
policymakers and the public as they seek 
to make sense of the complex relationships 
between data on technology and impacts 
on student learning. 



Good Numbers Are Hard to Find 

Computer and communication technologies 
pose many opportunities for improving our 
education system. To explore and evaluate 
that promise, we need data on how 
computers are currently being used and on 
the impacts of that usage. So far, there 
have been only a handful of systematic, 
representative, consistent data-collection 
efforts relating to the uses of computers in 
schools. It has proven difficult to interpret 
the results in order to answer important 
pedagogical, budgetary, and policy-related 
questions. 

The U.S. Congress was among the first to 
seek a better understanding of the use of 
technologies in public and private 
elementary and secondary schools. As 
early as 1982, the Congressional Office of 
Technology Assessment (OTA) published a 
report on "Information Technology and Its 
Impact on American Education". Three 
years later Congress asked the OTA to 
provide an overview of the use of 
computers in Chapter I and for limited 
English proficient students; this study was 
followed by the more comprehensive 1988 
report, Power On! New Tools for Teaching 
and Learning , a review of computer usage 
and issues in K-12 schools. Other 
educational technology studies followed, 
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on distance learning technologies for K-12 
education, on technology for adult literacy, 
and most recently, the April 1995 report 
on teachers and technology. 

In each of these studies, the first question 
asked by the Congress, not to mention the 
press and the general public, is one that 
NCES could appreciate— "What are the 
numbers?" As a member of the OTA staff 
for several of these projects, I soon 
learned that exact numbers were difficult 
to pinpoint. Much of our early data came 
from three sources: (1) the 1985 National 
Survey of Instructional Uses of School 
Computers, conducted by the Center for 
the Social Organization of Schools at Johns 
Hopkins University, under the direction of 
Henry Jay Becker, and summary 
newsletters from that source; (2) databases 
from the Curriculum Information Center of 
Market Data Retrieval, Inc.; and (3) 
databases from Quality Education Data, 

Inc. 

The data were difficult to compare due to 
different survey and sampling techniques, 
but we reported the numbers as we found 
them: somewhere between 1.2 to 1.7 
million computers for instructional use in 
K-12 public schools in 1988, growing to 
almost 5 million computers for 
instructional use in K-12 schools in 1995; 
10 states promoting distance learning in 
1987 but virtually every state using some 
form of distance learning by the fall of 
1989. 

The quality of current data is improving 
because commercial providers have found 
it valuable to those marketing technology 
to the K-12 sector. Several other firms 
and consulting groups make it their 
business to collect data on a regular basis 



on specific applications such as distance 
learning technologies and programs. 
Clearly, the market forces require accurate 
data for projections and purchases and for 
the planning of initiatives. 

However, as the data improve, they will 
continue to illustrate a fundamental point: 
information on the numbers of 
machines— computers, CD-ROMs, 
videodiscs, satellite dishes, 
telecommunications networks— while 
important, is not the key element. As the 
numbers grow, they are also taking on 
new meanings as the technology changes. 
These changes add new factors that help 
determine the impact that a computer may 
have in an educational setting: 

® The power and capabilities of the 
computers themselves are changing 
constantly. One cannot simply 
compare the numbers of older 8 bit 
machines with those that are 32 bit 
machines— the implications of 
power and speed are of magnitudes 
of difference. However, it is 
encouraging to note that recent 
surveys (e.g., Quality Education 
Data, 1996) also report on the 
numbers of computers capable of 
running multimedia applications. 

• Computers are being configured in 
new ways, with technologies that 
were once stand-alone now being 
incorporated into a single unit. We 
are no longer just counting separate 
PCs, separate hard drives, 
videodiscs, or projection devices; 
increasingly, the components are 
built in ways that confound counts 
but increase capabilities. 



• Networking further confounds the 
situation, as one machine can 
support dozens of students or 
applications at the same time. 

• The variety of software applications 
in use in the education setting is 
exploding; the nature and power of 
the applications is as significant as 
the numbers of machines 
themselves. 

Thus, unlike the relatively constant 
numbers collected on teachers and 
administrators in the SASS, and the well 
understood assumptions about what these 
numbers mean, numbers of pieces of 
technology are subject to far more 
complex interpretations. 

But Even Numbers Are Not 
Enough 

From a policy perspective, however, it 
was the information we could not find that 
became the most intriguing, and the most 
educationally significant, piece of the 
puzzle. In the Teachers and Technology 
report, for example, there were many 
questions Congress asked OTA to 
consider. How do state policies affect 
access to technology in schools? How do 
teachers use the technology that is 
available to them? What impacts do the 
technologies have? We contracted with 
experts in educational technology data 
analysis to help us find answers to some of 
these questions. Ronald Anderson of the 
University of Minnesota conducted a 
review of state technology activities related 
to teachers. This investigation, a 
telephone survey and review of state 
planning documents and guidelines, was 



helpful in providing at least a basic 
understanding of how various state policies 
affect access to technology, but it was not 
able to provide greater insights into the 
actual use of the technology. The survey 
looked at the following questions: 

(1) Does the state require or 
recommend that public schools 
integrate computer or information 
technology in the curriculum? (All 
but six states did.) 

(2) Does the state require public 
schools to offer computer-related 
courses such as keyboarding or 
computer literacy for students? 
(Twelve reported the requirement 
for students.) 

(3) Does the state have a mandate for 
computer competency or 
performance standards for students 
related to information technology? 
(Twenty reported mandating student 
computer competency.) 

(4) Does teacher certification in the 
state include a requirement for 
pre-service computer or technology 
training? (Nineteen states said yes.) 

(5) Does the state have a requirement 
for inservice computer or 
technology training? (Only two 
states reported such a requirement.) 

To assess whether the state requirements 
were in any way related to the numbers of 
computers in classrooms, Anderson looked 
at the survey data in relation to the state 
microdensity data reported by QED’s 1994 
report on Technology in Public Schools. 

We were surprised and frustrated to find 
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that greater state technology requirements 
did not necessarily mean more computers 
in the classroom (or vice versa). 

Anderson found a low of 8.1 students per 
computer in Wyoming (a state that only 
requires one of these factors (computer 
training for teacher certification); to a high 
of 22 students per computer in New 
Hampshire, a state in which three of the 
five policies are in effect (promoting 
technology integration in the curriculum, 
requiring computer course for students, 
and requiring computer training for teacher 
certification). This leads us to suggest that 
the relative amount of computer 
technology in a state should be used with 
great caution as an indicator of that state’s 
commitment to technology in instruction. 

What Do We Know about How 
Computers Are Used in Schools? 

The compelling questions still evade us: 
how much are educational technologies 
used, for what applications, by what kinds 
of teachers, under what conditions, and 
with what results? For the OTA study, we 
drew from a number of sources: case 
studies and site visits, conversations with 
teachers and administrators on site, by 
telephone, and at conferences; and 
traditional research reviews and Internet 
searches. We also commissioned a 
contractor report that reviewed existing 
surveys to see how their results might shed 
light on the questions of how much, how 
often, and in what ways teachers use 
technology. This was particularly 
challenging; as the author, Henry Becker 
of the University of California at Irvine, 
noted, "Unfortunately, much of the data 
needed for a complete picture of 



technology presence and use is simply not 
available". 

Becker found the best data in the 1992 
Computers in Education Study of the IEA 
(International Association for the 
Evaluation of Educational Achievement). 
Although their sample of schools was 
small (571 schools, with responses from 
computer coordinators) the IEA study 
asked questions about utilization, processes 
of decision making, and attitudes. 

Students reported their own computer 
experiences. Interestingly, the time 
reported for computer use was much 
higher when reported by computer 
coordinators (who reported that computers 
are used about 1 3/4 hours per student per 
week at the elementary level; two hours 
per student per week at middle schools, 
and approximately 3 hours per student per 
week at high schools) than when students 
themselves gave the estimates of computer 
usage (students reported using computers 
24 minutes per week in grade 5; 38 
minutes per week in grade 8; and 61 
minutes per week in grade 11). These 
discrepancies point out the need for greater 
sophistication in data collection methods. 

Regardless of the time spent using 
computers, the data suggested that the 
kinds of use are traditional. Indeed, the 
most common activities on computers for 
elementary students were drills in basic 
skills and instructional games. Also 
popular at all levels were general computer 
literacy activities and word-processing. In 
secondary schools, the data suggested that 
computers are used relatively infrequently 
for teaching and learning in traditional 
academic subjects, far less than in classes 
focused on teaching students about 
computers. It would be very useful to 



have more current data in this area to 
assess curricular impacts. 

NCES data are beginning to look at the 
role of technology within subject areas, as 
in the 1994 report, How School 
Mathematics Functions and in the data 
compendium for the NAEP 1992 
mathematics assessment of the nation and 
the states. One measure of barriers can be 
seen in teachers’ responses to the question 
regarding confidence in their preparation 
to teach: while 85 percent of grade 4 
students and 93 percent of grade 8 students 
had teachers who reported being "very 
well prepared" in mathematics concepts, 
only 15 percent of grade 4 students and 21 
percent of grade 8 students had teachers 
who reported that they felt very well 
prepared in computers. This kind of data 
should provide a red warning flag to 
policymakers if they expect to see greater 
computer usage in curricular applications. 

Four Issues Affecting the Use of 
Educational Technology 

While much of the information cited above 
is limited in scope, difficult to compare, 
and often anecdotal, its educational 
significance warrants further analysis. 
Despite the admitted limitations of this 
data, there are clear implications from this 
information that suggest issues that may 
provide a better basis for understanding 
technology use in the classroom. These 
issues include: the placement of technology 
in the school, the kinds of training and 
support teachers receive, and the value of 
technology for teachers in terms of the 
instructional goals and models the 
technology supports, as well as its role in 



increasing teacher productivity and 
professional development. 

1. Location of Technology Affects Use 

The location of technology within a school 
has a significant impact on use. It seems a 
basic point, but it still comes as a surprise 
to many educators: when computers are 
placed outside the classroom, it is less 
likely that they will be a part of the 
regular instructional day. In the past, 
schools typically aggregated computers in 
labs, in part to assure equal (if limited) 
access for all students and teachers. But 
lab time is time that must be scheduled, 
and it can create logistical nightmares that 
discourage use. Furthermore, a trip to a 
lab can be a "mini-field trip" requiring 
time to get to the lab, time to get settled, 
and time to close down and return to the 
"real class" on the other end, all eating 
into valuable instructional time. 

Nonetheless, the Becker study reported 
that most schools place a majority of their 
computers in computer labs-70 percent of 
all middle and junior highs had a majority 
of their computers in computer labs in 
1992. Only 35 percent of all school 
computers were in the classroom. Some 
schools are finding a way to offer more 
flexibility through use of laptop 
computers, kept recharged and placed on 
carts that can be rolled from room to 
room, moving the technology to the 
students rather than vice versa. But again, 
necessary scheduling and coordination 
create barriers to spontaneous or 
continuous usage. It’s a bit like having to 
share books, or schedule the use of 
pencils, if you are a regular computer 
user. 
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This same difficulty lies with 
telecommunications networks. Although 
the recent NCES study for the Department 
of Education reported substantial growth in 
the number of schools that have Internet 
access (growing from 35 percent to 50 
percent of all schools in one year), the 
number of instructional rooms with that 
access, while also improving, still remains 
low (increasing from 3 percent to 9 
percent in the same period.) When one 
realizes that "instructional rooms" includes 
libraries and media centers, it becomes 
clear that classroom access is even more 
limited. However, in considering 
deployment models for telecommunications 
systems in schools, cost factors are likely 
to lead policymakers to be tempted to 
continue to opt for the lab model. The 
recently released “Kickstart” report 
estimates that it could cost $11 billion for 
the one-time purchase and installation costs 
for a lab model (single room with 25 
computers, an Ethernet LAN in the lab, 
and 10 telephone lines); $22 billion for 
this configuration plus one computer and 
modem for each teacher; and $47 billion 
for a classroom model in which all 
classrooms have one computer for every 
five students, an Ethernet LAN across and 
within all classrooms, and a T-l 
connection. 

How much more are computers used when 
they are located in the classroom and are 
present in sufficient numbers for regular 
usage by a good portion of students? 

More research is needed to answer this 
question; however, data collected by high 
intensity programs like the Apple 
Classrooms of Tomorrow, the Buddy 
Project in Indiana, and the Christopher 
Columbus School in Union City, New 
Jersey, suggest that usage is much higher. 



However, other factors are also at play in 
these instances, notably substantial teacher 
training and support, new approaches to 
curriculum that support technology use, 
and greater flexibility of overall class 
organization and scheduling. It may be 
difficult to separate out the location factor, 
but it clearly plays a role that should be 
considered. 

2. Training for Technology Requires New 
Approaches and New Definitions 

Although most teachers have had some 
training about technology, far fewer have 
had suitable training to prepare them to 
use it in their teaching. For teachers to 
use technology effectively, they have to 
see the value it has for them. Technology 
use has to fit with their teaching style, and 
it has to work with the specific content and 
skills they guide their students to use. 
Furthermore, effective use must be based 
on visions of valuable applications, and be 
supported by training to develop skill and 
comfort with these applications. It also 
takes time and experimentation to adapt 
the technology into one’s routine. Only 
over a number of years do most teachers 
become "fearless" with technology. 

Timing. Timing of training is particularly 
important; when training is provided 
before the hardware or software arrives, or 
before teachers know what equipment they 
will be using, there is little opportunity to 
practice and apply that training back in the 
classroom. Classes are too often general, 
not focused on the level of expertise of the 
individual teacher or the teacher’s needs. 

It is ironic that we repeat the same 
mistakes in professional development as in 
the classroom: lecturing as if all learners 
are starting from the same point, rather 
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than realizing that teachers have unique 
experiences, understandings and needs 
which they bring to the class. 

Technical support. Finally, technical 
support is crucial: without assistance and 
trouble shooting for the inevitable 
problems that occur, teachers may be 
overwhelmed by frustration that can negate 
all their prior positive enthusiasm and 
desire to apply new technological skills. 

Teacher preparation. We also need better 
understanding of how new teachers are 
prepared to teach with technology. 

Finding little data in this area, OTA 
contracted for a review on this topic for 
the Teachers and Technology report. 
Although this was a limited survey with 
some methodological problems, it 
reinforced the message heard in case 
studies, interviews, and conversations with 
new teachers: technology is not central to 
the teacher preparation experience in most 
colleges of education. Although most 
teacher education students take some form 
of computer literacy course, they typically 
graduate without a clear vision of the ways 
technology can be used in their 
professional practice. Clearly this is an 
area of concern, especially when 
considering that 3.3 million teachers will 
be needed for K-12 schools by the year 
2003, 1.4 million more than are in the 
profession today. 

It is important to follow trends that affect 
teacher preparation, specifically state 
certification standards, and the 
accreditation for institutions of teacher 
education. For example, the National 
Council for Accreditation of Teacher 
Education Guidelines, developed by the 
International Society for Technology in 



Education in 1992, offer guidelines for the 
accreditation of educational computing and 
technology programs. These suggest that 
prospective teachers need to demonstrate 
knowledge and use of computers for a 
number of teaching activities. 

The ultimate criteria is, however, the 
marketplace; if districts and states will not 
hire new teachers who cannot demonstrate 
proficiency with technology in their fields, 
or, as is the case in Michigan, require 
that teacher candidates demonstrate 
knowledge of computer applications to the 
satisfaction of the school or district before 
that individual may engage in student 
teaching, education schools will no longer 
be able to avoid this responsibility. To 
date, however, only a limited number of 
states or districts have made this a 
requirement. 

As noted above, the Anderson study 
indicated that fewer than half the states 
(19) require training in computers or 
technology for all teachers seeking 
certification. Furthermore, this can vary 
from a semester course to a demonstration 
of the use of technology in teaching. 

More detailed data in this area would have 
a major impact on policy related to the 
preparation of new teachers. Instead of 
just asking, "How much training are they 
getting?" we should also be asking, "What 
kind of training is it, under what 
circumstances, and with what kinds of 
support? " 

3. Technology Should Be of Value to 
Teachers 

What has most often been missing in 
discussions of technology for education is 
a view of technology use that empowers 
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teachers by giving them reasons to use the 
technology to accomplish valued tasks. 

Few teachers have been encouraged to 
view new technologies as professional 
tools that can help them do their jobs 
better, more efficiently, or in new ways. 
Very rarely are teachers asked why they 
want to use the technology, much less 
given models that could guide this vision. 
Although most teachers believe in the 
value of students learning about computers 
and other technologies, many teachers are 
not aware of the resources technology can 
provide to them personally, as 
professionals, in carrying out the many 
aspects of their jobs. 

Our lack of attention to teachers’ needs is 
apparent in our almost total lack of data 
about the types of technology available to 
teachers. Little data have been collected at 
the national, state, or local level on the 
numbers of teachers (not just classrooms 
or students) who have a computer of their 
own at their desk, or a telephone, or a link 
to others via local area or wide area 
networks, or to the outside world via 
Internet access. Why is this figure not as 
important as the overall number of 
computers or telecommunications links 
within a school? I suggest that the fact 
that this question is not asked from the 
teacher’s perspective speaks volumes about 
the way teachers are treated as 
professionals within the educational 
establishment. 

Data from one study in Indiana, "A 
Computer for Every Teacher," give one 
window on possible outcomes. In this 
project participating teachers were given a 
computer and printer for their use at home 
or school, along with training and 
software. Teachers reported greater 



productivity (spending the same amount of 
time in class preparation and 
administration, but accomplishing more); 
professionalism (taking on the role of 
experts in certain areas, and sharing their 
successes with colleagues); and 
empowerment (pride in the school and 
their personal proficiency and 
accomplishments in an area that is valued 
in society). While job satisfaction and 
professional growth are difficult to 
quantify, they are areas of importance to 
staffing issues related to teacher turnover 
and continuing expertise. 

Indeed, one of the most powerful findings 
of the OTA study was the importance of 
technology for teachers for a variety of 
reasons. These include enhancing 
instruction that supports new models of 
learning, simplifying administrative tasks, 
and fostering professional growth. I will 
discuss each of these briefly. 

4. Enhancing Instruction; New Wavs of 
Teaching 

Teachers are becoming aware of the 
growing body of evidence on the value of 
technology for student achievement as 
measured by test scores, as well as its 
positive effects on student attitudes toward 
learning on students’ self confidence. 
Nevertheless, more contextualized research 
on the broad variety of educational effects 
of technology is necessary, in order to 
gain a clearer picture of the value added to 
instruction when technology is used. What 
is needed is research that gives a clearer 
picture of when, why, and how 
technologies impact student learning. 

We also need richer studies and better 
tools for measuring growth in skills 
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important for the information age if we are 
to develop better understanding of student 
growth in inquiry learning skills; in 
independent problem solving and 
collaboration skills; in written, spoken, 
and multimedia communication skills; and 
the ability to find, manage, and evaluate 
information from a number of sources. 
Other questions are equally important: 

How can we evaluate students’ confidence 
in themselves as learners? Can we get 
better measures of students’ appreciation 
of the outlook and traditions of other 
cultures or their development of moral 
values and the ability to empathize with 
others? How do we measure students’ 
love of learning and developing "habits of 
the mind?" 

There is also limited— but growing- 
evidence suggesting that technology use 
can create positive changes in teaching 
style. One important study looked at how 
“accomplished” computer-using teachers 
believe their teaching changed as they 
became more comfortable using 
computers. Teachers in this study 
reported that they spend more time with 
individual students, expect more from 
them, are able to present more complex 
material, are better able to tailor 
instruction to individual student needs, are 
more comfortable with allowing students to 
work independently and in small group 
activities, and spend less time lecturing or 
practicing or reviewing material with the 
whole class. We hear of computers 
making the teacher more the “guide on the 
side” rather than “sage on the stage”, 
more a coach or facilitator than 
all-knowing expert, but we need solid 
studies to confirm this realignment of 
teaching style. As school reform literature 
calls for greater attention to student 



inquiry, authentic challenging tasks, 
collaborative learning, and 
multidisciplinary curriculum, it will be 
important to understand if the use of 
technology does indeed support these 
goals, and if so, how and why? 

For example, technology offers valuable 
assistance to teachers struggling with 
demands created by performance 
assessment. Keeping track of rich but 
extensive histories of student performance 
over time can be a real challenge. 
Teachers are finding ways to use video to 
record performance, personal digital 
assistants to keep track of their 
observations of student activities, and 
electronic portfolios to collect and 
maintain student work on disk. These 
technologies allow records to be retrieved 
and updated to demonstrate cumulative 
student progress. Technological 
applications like these make it easier for 
teachers to adapt to some of the new 
student-centered assessment approaches 
that educational reformers are requiring. 

Much is needed in the form of additional 
surveys and in-depth case studies and 
analyses to develop a better understanding 
of the dynamics of technology’s part in 
changing teaching style in these and other 
areas. 

Tools for greater teacher productivity . One 
of the most obvious benefits of computers 
for teachers is the streamlining of the 
myriad administrative tasks that take up so 
much of teachers’ time. Attendance 
records, grades, book lists, classroom 
inventories-these and hundreds of other 
time-consuming tasks can be streamlined 
or automated with a personal computer. 

To use the tools effectively, however, 
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teachers must be given the release time to 
learn the appropriate software in order to 
set up and maintain files. 

Teachers also report they can access more 
resources for preparing and updating 
lessons through telecommunications 
networks, using materials that are more 
current and more compelling than what is 
found in standard textbooks. They can 
also add their own materials to share with 
others. Curriculum development is a 
tedious process; however, using 
collaborative software teachers can work 
together in creating new materials, 
drawing from the resources each can 
provide. Data from sources like the 
Eisenhower Clearinghouse electronic 
catalogue of instructional plans written by 
teachers in mathematics and science can 
give an idea of the popularity of this 
activity among teachers. 

Perhaps one of technology’s greatest 
selling points is the way it offers 
opportunities for greater communication 
with parents, whether the technology is a 
simple phone in the classroom supplied 
with a voicemail account, or more 
sophisticated school/home 
telecommunications links that allow 
exchange of e-mail messages. With 
research showing parental involvement as 
one of the greatest indicators of student 
success in schools, any tool that helps 
teachers encourage that involvement also 
becomes a productivity tool for the 
teacher. 

Tools to support teachers ’ learning. As 
states encourage new forms of instruction, 
changes in content, and new ways to 
measure success, it is important to develop 
a better understanding of how technologies 



can serve teachers in meeting these new 
challenges. Certainly the new curricular 
standards are going to have a major impact 
on the training needs of teachers. Just as 
the subject matter content and pedagogy 
recommended by the National Council of 
Teachers of Mathematics has created a 
huge need for helping teachers learn to 
adapt their teaching according to these 
standards, similar national standards in 
science, history, and English will create 
training demands in these curricular areas. 
And, as cross-curricular learning is 
emphasized, teachers will have to become 
more comfortable with content and 
activities outside their specialty field. 

Education can take lessons from business 
in this regard. Industry has found it much 
less expensive, and more effective, to use 
a "training on demand" model, in which 
education and information is brought 
directly to the worker at his or her 
workstation. For example, by using an 
interactive satellite network rather than 
bringing a "dog and pony show" of 
courses to 12 different cities for 4-5 weeks 
per quarter, Hewlett Packard has cut its 
sales training costs from $2 million to 
$200,000 per year. The sessions now take 
two days, require no travel, and provide 
training the employee can use 
immediately, to improve retention, recall, 
and application. 

Applications like this could change 
in-service training for teachers and greatly 
enhance options and flexibility. For 
example, video tapes of teaching based on 
NCTM math standards form the basis for 
the PBS-sponsored Mathline series; the 
tapes are shown over the local PBS station 
and can be reviewed as needed by 
participants as they study the concepts and 
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content within them. They are 
supplemented by online group discussions 
of the tapes in which teachers share their 
experiences in trying the approaches and 
exercises back in the classroom, reflecting 
on what works, what doesn’t, and why in 
a supportive, collaborative forum. 

This blend of formal training and informal 
support could be the professional 
development model of the future, giving 
more individualized, just-in-time training 
when and where teachers need it. With 
teachers adding their own content and 
experiences, they become supporters and 
mentors to their colleagues near and far. 
The informal support of peers, as well as 
the access to resources and experts 
anywhere in the world, is likely to have a 
profound effect on teachers and teaching, 
not to mention student learning. 

Although the benefits seem obvious, there 
has also been little research on how 
networking is used by teachers, the value 
of the informal training it provides, the 
cost savings it offers, or how much 
teachers value this form of support. Does 
teacher collaboration increase? In what 
ways is this of value to teachers? How 
does this affect their image of themselves 
as teachers, as learners, as professionals; 
and what impact does this have on 
motivation and aspirations? 



Final Recommendations for NCES 

As noted earlier in this paper, the 
expenditures in technology at the K-12 
level, some $2.4 billion a year, warrant 
policymakers taking a closer look at how 
much and how well that technology is 
used. While these measures may not fit 



classic cost/benefit models, they should be 
set in terms that resonate with 
policymakers. Do teachers use their time 
in ways that are more efficient? Do 
students have more time for learning, and 
is the kind of learning that goes on 
important learning? Can expenditures in 
other areas that consume large portions of 
education budgets-such as materials and 
textbooks— be cut back as technology 
expenditures increase? Are staffing costs 
affected when technology resources 
substitute distance teachers for some 
courses, when more teacher development 
is done informally rather than in formal 
courses, or when support is provided by 
student technology aides or local 
businesses models like the US Tech. 
Corps? Are the continuing costs of 
technology expenditures (for hardware, 
software, networking, training, and 
support) worth the drain on otherwise 
strapped budgets? 

While these questions are beyond the 
scope of the SASS, they are areas of 
policy concern at all levels and should be 
the basis for comprehensive surveys, fast 
response surveys, and longitudinal studies 
that can inform these decisions. As one 
reviewer suggested, "A useful plan for the 
School and Staffing Survey would be to 
link the descriptive data collected in that 
survey with simultaneously funded 
research that more directly addresses the 
causal factors that account for variation in 
teacher practices and student outcomes. 
This coordinated two-pronged approach 
will help us to understand how to realize 
the potential of educational technology to 
assist in improving teaching practice." 

One can be encouraged by the other papers 
in this series, suggesting how the ability to 
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collect and share data electronically make 
more complex surveys perhaps more 
feasible today. I hope that the planning 
done by NCES for future studies will take 
these important issues into account and 
that valuable NCES resources can be 
leveraged to the greater benefit of 
educational improvement. 
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LINKING STUDENT DATA TO SASS: WHY, WHEN, HOW 

Phillip Kaufman, MPR Associates, Inc. 



This paper considers the feasibility of 
linking a student data sample with the 
SASS teacher and administrative data. 
NCES has from time to time considered 
linking their student-based elementary and 
secondary surveys to the school- and 
teacher-based surveys. These thoughts 
have usually been centered on the 
analytical power that such a 
student/teacher data set would hold. 
Budgetary concerns— in terms of both 
fiscal program budgets and burden 
budgets— have also been important, but the 
analytical justification of linking student 
data to teacher data has generally taken 
precedence. However, speculation on the 
feasibility of linking datasets is of 
particular importance now in the current 
climate of budgetary constraints and 
distrust of federal data collection among 
segments of the public. 

NCES (and indeed all of the governmental 
statistics community) is entering an era 
when hard choices need to be made 
concerning data collection and reporting. 

In an era when we face increasing 
demands for more and better data from a 
wide variety of educational policymakers 
and researchers, we are also being asked 
to do more with fewer resources in terms 
of both program money and personnel. 
Thus it may no longer be feasible to 
collect data on schools, teachers, and 
programs through the Common Core of 
Data and the Schools and Staffing Survey, 
conduct another National Assessment, 
continue to track the early childhood 
cohort of students (in ECLS), and launch a 



new longitudinal study of high school 
students. If it is not feasible to collect all 
of these data as they have been collected in 
the past (regardless of their analytical 
merit), it seems to me that ways must be 
found to collect all of these data with new 
methods or systems. Unless we think hard 
about these issues, opportunities may be 
lost and we will create gaps in our 
knowledge about American schools and the 
impact of the last few decades of reform. 
Old ways of conducting the business of 
data collection may have to be modified in 
light of the increased budgetary constraints 
imposed by Congress and the President 
and the simultaneous increase in analytical 
demands of the public. 

Some Ground Rules 

In this paper I discuss the rationale for 
linking a student data collection in SASS 
and then explore several options for 
collecting these data. However, before 
launching into the main body of the paper, 
I would like to lay out a few principles to 
organize my discussion of these issues. 
These guidelines have to do with (1) what 
dimensions of student data should be 
collected, and (2) what criteria should be 
used to judge the merit of the proposed 
new data collection system. 

Dimensions of student data. As I will try 
and develop further in this paper, it seems 
to me that at least three aspects of students 
are important to track and should be a part 
of any system of student data. While 
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perhaps not necessary elements of a SASS 
student component, these elements should 
be (and are) part of the overall data 
strategy for NCES and should be 
considered when thinking about what kind 
of student data should be attached to 
SASS. These elements are: overall 
academic performance, growth in 
achievement, and successful transition into 
and through the increasing demands of 
schooling and work. 

Among these three, measuring overall 
academic achievement is perhaps most 
important (for why else can we justify 
public and private investment in schools) 
and has traditionally had the most 
attention. Both NAEP and the longitudinal 
studies have made estimates of overall 
achievement levels of various groups of 
students over time. Accurately measuring 
growth in achievement (especially in 
observational/survey data) is perhaps the 
most challenging. This has been done by 
analysts using both HS&B and NELS:88. 
Examining critical transitions has 
historically received the least attention, but 
has come under increasing scrutiny as the 
educational community has realized the 
importance of studying the life-course and 
its impact on education (Pallas, 1993, p. 
20). It seems to me that three main 
transition periods are important to keep in 
mind while considering student data within 
SASS: (1) the passage from middle school 
to high school, (2) the path through high 
school to graduation, and (3) the transition 
from graduation to school or work. 

Evaluative criteria. While keeping these 
three elements of student data in mind, I 
need to set up a few criteria to judge the 
worthiness of any proposal to linking 
individual data with SASS. It seems to me 



that for such a merger to make sense, it 
must pass at least one of two tests. First, 
it must make sense in terms of efficiencies 
of administration or respondent burden. 
That is, it must prove to save budgetary 
resources (either program budget or 
burden budget). Secondly, it should make 
analytical sense. That is, the merger 
should result in a data system that allows 
more comprehensive and sensible 
inferences to be drawn. 

To summarize, I start this discussion 
considering three main elements of student 
data and two principles of an adequate 
argument for linking student data within 
SASS: 

Elements of a student data system: 

• Overall academic performance, 

• Growth in achievement, and 

• Successful transition through the 
increasing demands of schooling 
and work 

Criteria for judging the adequacy of 
including a student component in SASS: 

• The merger should produce some 
cost benefit, and 

• The merger should engender an 
analytical payoff 

All of the above must also be considered 
in the context of the mission of NCES (as 
I paraphrase it): (1) providing to the public 
accurate information on the “Condition of 
Education,” (2) producing policy-relevant 
but policy-neutral research reports on 
current and/or enduring issues in 
educational policy, and (3) providing 
databases that other analysts can use as 
research tools in their own policy work. 
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The data needs differ for these three 
functions— ranging from fairly descriptive 
data for function 1 to data for function 3 
with the potential for multivariate analysis 
and “cautious causal” analysis. 



Why Collect Individual Student 
Data through SASS? 

Much of the data that NCES collects are 
not on individual students, but are on 
characteristics of schools and other 
educational institutions. For example, 
SASS now collects data on school districts, 
schools, principals, and individual 
teachers. Detailed information is available 
on characteristics of the curriculum, 
qualifications of teachers, school and 
district level policies and practices. 
Traditionally, what student data have 
existed in SASS were generally aggregated 
to the school level before being captured. 
For example, percent of students receiving 
free lunch, percent of students of various 
racial-ethnic groups, etc., have been 
attached to the school files since the first 
cycle of SASS. 

However, while it is important to be able 
to accurately measure and track schools, 
teachers, and curriculum practices, these 
data provide the context for measuring the 
main component of education— student 
achievement, growth, and progress. As 
the nation tries to assess and track the 
implementation of school reform, the data 
on schools and teachers do provide 
valuable indicators of the extent of 
reform— and these data have been used 
effectively over the last decade for this 
purpose. However, these data are much 
more policy relevant when used in the 
context of seeing how they are related to 



individual student achievement, growth, 
and experiences. 

It is possible to attach to SASS additional 
aggregate measures of student 
characteristics and outcomes. In fact, this 
is the approach advocated by Don 
McLaughlin in his response to an earlier 
version of this paper. McLaughlin makes 
the case for aggregate data based on the 
tremendous improvements in the 
assessment practices of many state 
departments of education. He advocates 
using these state assessment data 
(presumably available for each school in 
SASS) by linking them to the state NAEP 
assessment. Dave Thissen has conducted 
such equating for the North Carolina state 
assessment. 

I appreciate McLaughlin’s contribution to 
this discussion and his comments on using 
state assessments are well reasoned. 
However, as he acknowledges, cross- 
sectional data on student outcomes are less 
interesting than longitudinal data (and, in 
my opinion may not be worth the effort of 
collecting at all). Collecting longitudinal 
aggregate data on student achievement 
within schools is of more interest, but 
(again in my opinion) not as useful as 
collecting individual student data. That is, 
aggregate test scores or mean outcomes do 
not capture the individual variation in 
achievement that traditionally has been of 
such interest. 

For example, the variance of test scores 
within schools has been used as an 
outcome measure in assessing the 
effectiveness of schools. High mean test 
scores may be due to the school’s efforts 
at increasing the learning of students 
already achieving at a high level or may be 
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due to the school’s successful attempts at 
raising the scores of students at the bottom 
of the ability ladder. Mean scores mask 
these important differences in the impact 
of school policies and practices. 
Furthermore, the transitional experiences 
and out-of-school experiences are so 
important to those educational outcomes. 
Therefore, while it may be worthwhile for 
SASS to explore the possibility of 
attaching aggregate and longitudinal 
assessment data to their regular data 
collection, it seems to me that still 
accurate estimates of the associations of 
teacher and school characteristics on 
student outcomes necessitates the linking in 
some way of individual students (or 
similar groups of students) with individual 
schools, teachers, or policies (or similar 
groups thereof). 

Of course, an expansion of the current 
SASS student survey (based on 
administrative records of students of 
sampled teachers) could add immeasurably 
to the analytical power of SASS. This 
option would build on the current efforts 
to include student data in SASS. While 
current student samples would have to be 
increased to be representative of the 
school, it still seems reasonable that this 
would be the most cost effective choice. 
However, it is perhaps the least effective 
analytically. Only limited kinds of data 
could be collected by administrative 
records — race-ethnicity, sex, absences, 
maybe grades. Test score data that would 
be comparable across schools would not be 
available. Furthermore, while data on 
dropout status may be available from 
administrative records, we have known for 
a long time that these data are unreliable 
as indicators of student status. They may 
be reliable indicators of what that school 



thinks is the status of the student, but that 
student may have enrolled in another 
school (perhaps an alternative school) or 
may have taken the GED and received an 
alternative credential. Student data would 
also be cross-sectional and vulnerable to 
all of the weaknesses of cross-sectional 
data. 

Thus, attaching only individual 
administrative student statistics to school 
and teacher data would miss invaluable 
insights that are derived from observing 
student outcomes and transitions in the 
context of student’s prior experiences, 
aptitudes, and ability levels in school — data 
that can only be measured through 
individual student surveys. Administrative 
data also would fail to capture or measure 
the impact of the transitions that students 
make through different schools and 
classrooms to the world of work and 
family life. Clearly, while collecting 
student data through administrative records 
may be cost effective, they do not provide 
the kind of data that add as much to the 
analytical power of SASS— only individual 
student data can do this. 

Over the years, NCES has relied primarily 
on two vehicles for collecting data on 
individual students— the National 
Assessment of Educational Progress 
(NAEP) and the system of longitudinal 
studies including the National Longitudinal 
Study of 1972 (NLS:72), the High School 
and Beyond study (HS&B), and the 
National Education Longitudinal Study of 
1988 (NELS:88). As an integral part of 
these data collections, individual student 
data have been directly linked to data 
about the student’s teacher, classroom, and 
school. NAEP and the longitudinal studies 
accomplish this by including school and 
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teacher questionnaires along with student 
background and assessment data. Data on 
student outcomes can therefore be linked 
with data on educational context. 

However, much of the school, teacher, 
and classroom data collected by the 
student-based surveys are collected in 
more breadth and depth in SASS, or in 
any case is redundant with data collected 
by SASS. Furthermore, SASS collects 
data about schools, teachers, and, most 
importantly, school districts that are not 
collected by NAEP or the typical 
longitudinal study. In a time of tight 
budgets (that may become even leaner) a 
reasonable question is why not borrow the 
strengths of both types of surveys and link 
the more detailed student data NAEP or a 
NELS to the richer teacher, school, and 
district level data in SASS? In this manner 
each may provide contextual data to better 
interpret the other and possibly reduce the 
overall respondent burden (although 
perhaps increasing the burden on those 
sampled)— thus fulfilling the requirement I 
set for myself in the introductory section 
of this paper. This is the topic to which I 
will turn next. 



Linking NAEP to SASS 

Advantages of a Linkage with SASS 

NAEP has several distinct advantages over 
a NELS in such a linkage. The primary 
advantage is in the content detail that is 
provided in the assessment and the age or 
grade coverage available in NAEP. Due 
to an adaptation of matrix sampling called 
balanced incomplete block (BIB) spiraling, 
the design of NAEP allows for broad 
coverage of curriculum content while 



minimizing the burden to individual 
students. For example, while no student 
takes all test items, the 1992 NAEP 
mathematics assessment contained 178 
items at grade 4, 205 items at grade 8, and 
201 items at grade 12. This allows 
reliable estimates across five content areas 
in mathematics as well as three ability 
areas. (The mathematics assessment in 
NELS: 88 in contrast, contains only 40 
items and five proficiency levels.) 

NAEP also includes a student 
questionnaire that solicits background 
information on each student. NAEP is 
built to obtain good estimates of 
proficiencies in a variety of areas for 
groups of students. One of the primary 
strengths of NAEP is its ability to track 
the overall achievement levels of U.S. 
students over decades of time. From the 
early 1970s NAEP has reported on the 
mathematics and reading achievement of 
elementary, middle school, and high 
school students. This has provided 
educational policymakers and the general 
public with an immeasurably valuable tool 
in monitoring the health of our educational 
system. 

Weaknesses of a Linkage with SASS 

While NAEP has some obvious strengths 
as a candidate for merger with SASS, it 
also has several weaknesses. Those 
aspects of NAEP that do not lend 
themselves to a merger with SASS are 
analytical more than procedural. For 
example, the main weakness of NAEP is 
that it is not longitudinal. Merging a 
cross-sectional SASS and a cross-sectional 
NAEP would still result in a cross- 
sectional survey. While the cross-sectional 
design of NAEP allows for rich data for 



descriptive indicator work, the merged 
dataset with its rich contextual data and 
assessment data would still be of little use 
in producing valid analysis of the 
association of school policies and 
practices. In fact, the existence of such a 
dataset may actually encourage “invalid 
but potentially influential studies of schools 
effects that could seriously distort 
policy .” 1 That is, secondary analysts (or, 
with due apologies to William Raspberry, 
a columnist looking at published NAEP 
reports) could make erroneous conclusions 
about school policy based on the real but 
misleading associations in the data. 

Another analytical weakness of NAEP is 
that it does not contain good measures of 
student socioeconomic status 2 (and may 
never contain such measures). Without a 
measure of this kind, it is difficult to 
accurately describe the contribution of 
school process and policy variables on 
student outcomes. Most of these process 
variables are related to student 
socioeconomic status and/or student body 
socioeconomic status. Again, invalid but 
persuasive inferences could be drawn from 
these data. 

However, while socioeconomic status is a 
prominent gap in the student background 
variables provided by NAEP, it is only 
one of several variables that one would 
want to collect and measure in order to 
make satisfactory inferences from 
associations found in the data between 
achievement levels and school 
characteristics and practices. These 
variables include, but are not limited to, 
self-concept, attitudes toward school, and 
peer group attitudes and opinions. 



As mentioned above, one of the major 
contributions of NAEP is the trend data 
that it provides on student achievement in 
the United States. This strength of NAEP, 
however, proves to be one of the greatest 
arguments for not linking it to SASS. It 
seems unreasonable to expect that such a 
linkage could be done without some 
modification of the design of 
NAEP— either in its sampling design or its 
administration design. Such changes in the 
design of NAEP could result in changes in 
the estimated proficiency levels in the 
United States . 3 In addition, SASS is a 
fairly new and dynamic dataset. Again, 
given the importance of the NAEP time 
series, one would want to be very cautious 
in any changes to the design of SASS that 
would effect the design of NAEP, in either 
content or sampling design. Therefore, 
locking the design (and administration) of 
SASS to NAEP would make future 
changes in SASS very difficult. For 
example, currently NCES data collections 
poorly measure the classroom experiences 
of students. That is, while being able to 
describe educational inputs— students, 
teachers, schools — they do not measure 
educational processes well— what actually 
goes on inside the classroom. There 
would be many issues in incorporating a 
sample of classrooms within the design of 
SASS— including preserving the trend data 
of schools and teachers from earlier rounds 
of SASS. Adding the encumbrance of 
ensuring that the trend data from NAEP is 
also preserved would make this task even 
more difficult. 

Furthermore, while merging the two 
surveys could produce savings in total 
respondent burden to the educational 
system, it almost certainly would increase 
respondent burden for individual schools 
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and teachers that are sampled in the 
merged survey system. This could result 
in lower response rates and threaten the 
data quality for both surveys. NAEP has 
traditionally relied on high response rates 
to ensure the quality of the trend data. 
Again, in my opinion the integrity of these 
data is too important to jeopardize in a 
SASS/NAEP merger. 

The NAEP emphasizes the production of 
reliable estimates of national and state 
achievement levels. Consequently, NAEP 
does a good, but not perfect, job of 
estimating the first element of student data 
I outlined above— measuring overall 
student achievement. However, the 
strength of NAEP is in measuring 
aggregate-level measures of proficiency 
and not individual or school-level measures 
of proficiency. The capture of individual 
proficiencies or achievement levels has 
never been the main goal of NAEP. 

Given the complex nature of the plausible 
value methodology, individual or small 
group proficiencies are measured with a 
good deal of measurement error. 

NAEP is also a survey that emphasizes 
content depth over breadth of background 
variables. The burden budget of NAEP 
goes into accurately measuring content. 
Student background coverage is not 
ignored, but certainly has less emphasis 
than in the longitudinal studies. The 
longitudinal studies, on the other hand, 
have had somewhat different goals. For 
example, while NELS:88 also aspired to 
provide accurate estimates of group 
proficiencies, it had the added burden of 
obtaining accurate estimates of school and 
individual level proficiencies and 
individual growth. There was also the 
emphasis in NELS:88 on the measurement 



of a variety of student educational 
outcomes and not just academic 
achievement. To control respondent 
burden, the academic assessment tools in 
NELS:88 had to be much shorter in scope 
and content than the NAEP assessment. 
NAEP puts its burden dollars in the depth 
of the content while NELS:88 put its 
burden dollars in breadth of outcomes and 
background information. 

Furthermore, because NAEP does not 
measure students longitudinally, it does not 
do a good job of measuring (and does not 
attempt to measure) the other two elements 
of my list of student data above — growth 
in achievement, and successful transitions 
through the increasing demands of 
schooling and work. Longitudinal studies 
are needed to track these types of 
outcomes. For these reasons a new NELS 
(or some modification of NELS) may be a 
better candidate for merger with SASS. It 
is to this topic that I turn next. 



Linking a New NELS with SASS 

While it is important to measure and track 
overall achievement levels, it is also 
important to be able to associate 
differences in school policies and practices 
with student achievement. It is almost 
impossible to make valid inferences about 
the impact of school policies with cross- 
sectional data— regardless of how rich the 
individual data may be. Of course, 
making clear inferences about these kinds 
of associations is done best by experiments 
in which students are assigned to 
educational treatment conditions and 
subsequent growth in achievement is 
measured (Metcalf, 1995). 
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However, true experiments in education 
are difficult to conduct and maintain under 
the best of circumstances. Many 
educational researchers have therefore 
relied on observational survey data to 
make cautious inferences about policy 
effects on achievement gains. While these 
studies have many well known inherent 
flaws, most educational researchers and 
policymakers have been determined to not 
let the “perfect be the enemy of the good” 
and have conducted well thought out and 
executed policy studies with the 
longitudinal studies data systems provided 
by NCES (Heyns & Hilton, 1982, pp. 
89-102). 

Three Options to Consider 

It seems to me that there are at least three 
options to attaching a longitudinal student 
component to SASS. These are outlined 
below. 

(1) Attach student administrative data to 
SASS and return to those schools to 
pick up longitudinal data. This 
option would be substantially more 
expensive than simply attaching 
student administrative record data 
to SASS since one would have to 
return to the SASS schools to 
follow up on the students sampled 
in the first year. SASS is currently 
on a five-year cycle. Presumably 
one would want to go back to 
recapture student data on a more 
frequent follow-up schedule- 
— perhaps every two years. Re- 
surveying schools every five years 
to follow up on students is perhaps 
too long a periodicity to make 
timely estimates of student 
outcomes. One could of course go 
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back to the SASS schools (or 
sample of SASS schools) to capture 
just those administrative records 
that one needs. However, even this 
would increase the administrative 
and respondent burden of the 
survey system without providing 
much in the way of analytical 
payoff. Student test data would 
still not be available and 
consequently measures of growth in 
achievement would also not be 
available. In terms of measuring 
transitions, one would know if 
students were still enrolled in that 
school, but would know precious 
little else about the students’ 
transitions to other school or work. 
Furthermore, some portion of the 
students would have moved, 
making follow-up of their status 
difficult and expensive. 

In addition, learning takes place in 
an interaction of school, home, and 
family. A student data collection 
based solely on school records 
obviously records only one aspect 
of this learning system. The 
longitudinal studies have long 
recognized this and have tried to 
measure the other aspects of the 
student’s learning environment. 
Measuring only one component 
does not allow one to fully examine 
the totality of the students’ learning 
experience and how the different 
components interact with one 
another. 

(2) Create a new longitudinal survey and 
“link” several items to SASS items. 
NCES could field a new NELS 
with either an eighth-grade or 
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tenth-grade cohort and use identical 
items from SASS in its school and 
teacher questionnaires. Linking 
these data would provide some 
analytical payoff in terms of 
generalizabilty of the data provided. 
It would also decrease the burden 
to individual sampled schools, 
which would presumably not have 
to respond to the both the SASS 
and NELS survey instruments. 
However, it would increase overall 
response burden and would likely 
increase overall administrative 
costs. The analytical payoff would 
also be somewhat weak, since the 
linked data to SASS would not 
include all of the contextual data 
provided by the new NELS. 

(3) Merge a new NELS with SASS. 

NCES could field a new NELS in a 
sample of SASS schools. For 
example, the 1998 SASS could 
become the base year of NELS: 98. 
The overall analytical reward of 
such a merger could be substantial. 
This class of students will be on 
schedule to graduate in 2002, thus 
leading to clean comparisons 
among the high school classes of 
1972 (NLS-72), 1982 (HS&B), and 
1992 (NELS-.88). The longitudinal 
studies have traditionally have had 
teacher and school data, but have 
not have had district-level data to 
attach to student data. 

Furthermore, the richness of the 
SASS teacher and administrator 
data would enhance the student and 
parent data from NELS. Student 
assessment data (perhaps both 
cognitive and affective) could be 
attached to the SASS data to enable 



analyses of the association of 
outcomes data with school and 
district policy information. Data 
would also be collected with 
several follow-ups and would thus 
be able to measure growth in 
outcomes. Information would also 
be available to track the success of 
students in making critical 
transitions through school and 
work— for example, transitions 
from middle school to high school, 
through high school to high school 
completion, and from high school 
completion to postsecondary 
education and/or the world of 
work. 

While a new NELS attached to SASS 
makes sense analytically, it also makes a 
great deal of sense in terms of cost 
savings. The SASS data collector will 
have already contacted the schools and 
collected data from districts, schools, and 
teachers. A new NELS would only have 
to supplement these data with a student 
and parent questionnaire— the teacher and 
school data would be collected within the 
normal SASS administration. Using the 
1998 SASS survey as the base year of a 
new NELS has been shown to indicate a 
substantial cost savings over a separate 
sample design (J. Owings, internal memo, 
1995, National Center for Education 
Statistics). 

While total response burden would 
presumably be decreased by a NELS/SASS 
merger, the burden to individual schools 
will almost surely increase. However, this 
increase in response burden would have 
the potential to effect the response rates of 
the NELS data collection effort rather than 
SASS. SASS should not have to pay any 
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part of the response rate price associated 
with the merger. 

Thus, a new NELS attached to SASS 
would meet the requirement that I set forth 
in the introduction to this paper. It would 
collect all three types of data that I think 
are important — overall achievement data, 
data on cognitive and affective growth, 
and data on critical transitions. It would 
also meet the two criteria for a reasonable 
merger— it would make sense analytically, 
and it would make sense economically. 

However, a new NELS attached to SASS 
would still have to overcome several 
obstacles and several issues will need to be 
addressed in designing a new NELS. In 
fact, fleshing out a design for a new 
longitudinal study attached to SASS 
deserves its own design conference. 
However, short of this, I briefly outline 
two areas of concern in the next section. 



The Design of the National 
Longitudinal Study of 1998 

What age cohort should NELS:98 begin with ? 
To track the transitions I outlined above, 
NELS: 98 could start with either an eighth- 
grade cohort (to follow the transition from 
middle school to high school and allow 
trend comparisons with NELS: 88), with a 
tenth-grade cohort (to follow the transition 
from high school to graduation and allow 
trend comparisons with HS&B and 
NELS: 88), twelfth-grade cohort (to track 
the transition from high school to 
postsecondary education or work), or some 
combination of the above. 

Starting with another eighth-grade cohort 
has a lot of analytical appeal. The 



transition from eighth grade to high school 
is a significant passage. Meaningful 
research has been done with the NELS: 88 
cohort on this issue. Furthermore, data 
from NAEP and from NELS: 88 indicate 
that a significant amount of cognitive and 
academic growth occurs during this 
period. Larger gains are realized, on 
average, between the eighth and tenth 
grades than between the tenth and twelfth 
grades (Crouse & Ralph, 1996). 

However, despite the intuitive appeal of 
starting with an eighth-grade cohort, for a 
variety of reasons a tenth-grade cohort 
may be more feasible at this time. The 
primary reason for this is the ease with 
which tenth-grade students can be followed 
and therefore the lower cost involved. 
While younger cohorts are perhaps always 
more desirable analytically than older 
cohorts, following younger cohorts is 
always more expensive than following 
older cohorts. For example, almost 90 
percent of NELS: 88 eighth graders 
changed schools between the eighth and 
tenth grades, while less than 20 percent of 
NELS: 88 tenth graders changed schools 
between the tenth and twelfth grades. 
Tracking students from the eighth to the 
tenth grade proved to be much more 
expensive than originally estimated with 
the NELS: 88 first follow-up study. 

Furthermore, while there was great 
analytical payoff to estimating the growth 
in achievement of an eighth grade in 
NELS: 88, the complexities of the 
psychometrics involved in this effort were 
severe. Because the NELS: 88 test battery 
was used to measure overall achievement 
levels and growth between the eighth and 
twelfth grades, floor and ceiling effects 
were much more worrisome that in HS&B, 
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where growth was measured between the 
tenth and twelfth grades only. The 
resulting adaptive nature of the NELS:88 
assessment created analytical problems 
with researchers not sophisticated with 
psychometrics. For example, measuring 
gains in mathematics proficiency was 
much more complicated than merely 
looking at IRT gains scores, as had been 
done in HS&B. Since different kids took 
different tests, gains had to examined in 
terms of gains in proficiency functioning 
rather than raw or IRT estimated gains. 
Again, this complication was due to the 
fact that the assessment instruments had to 
have a multilevel design to guard against 
the floor and ceiling effects that could 
occur when testing spanned the eighth 
through twelfth grades. 

It is also interesting to speculate whether a 
twelfth-grade cohort (either selected on 
their own or an “aged” tenth-grade cohort) 
could be attached to SASS in the high 
school years and then attached or merged 
in a new Beginning Postsecondary Student 
(BPS) survey when the year after they are 
scheduled to leave high school. I realize 
that the sampling issues here may be 
enormously complicated and can only 
speculate about the complexities of such an 
overlapping or multiple frame design. 
However, by designing the three surveys 
in this manner, one would have the 
merged power (and savings?) of a SASS, a 
NELS, and a BPS. 



Periodicity of SASS 

To parallel the structure of the HS&B and 
NELS: 88, the new longitudinal study 
should be on a two-year cycle. That is, if 
NCES starts with a tenth-grade sample, 



they would want to go back and re- 
interview the sampled students two years 
later when most of them will be in the 
twelfth grade. In this manner, trend 
analyses could be run with the HS&B and 
the NELS: 88 tenth- to twelfth-grade 
cohorts. Since SASS is currently on a 
five-year cycle, the two-year follow-up 
would have to be done separately from the 
normal SASS cycle. These independent 
follow-up interviews could be done either 
as a CATI or as in-school interviews. In- 
school interviews would probably be more 
costly, but would be more efficient if 
cognitive assessments were conducted 
during this follow-up. (Unless someone 
develops a way to efficiently do a 
NELS: 88 comparable assessment through 
CATI.) 

Furthermore, in many ways HS&B and 
NELS: 88 were multiple-cross-sectional 
datasets. Data were collected on the same 
people for two years apart. What went on 
in between those two data points is often 
hard to determine. For example, detailed 
information on school enrollment has been 
difficult to obtain from HS&B and 
NELS: 88. One knows from the various 
follow-ups if sampled members were 
attending school at the time of the follow- 
up, but do not know much about their 
enrollment status in between the follow-up 
survey dates. One could use CATI to 
efficiently go back to these students more 
frequently than a two-year cycle and 
collect such time-sensitive data. These 
intermediate interviews would be limited 
to just a few items (dropout status, 
pregnancy status, employment status) with 
fewer time dependent variables reserved 
for the more in-depth two-year follow-up 
survey. 
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Summary 

The argument for attaching a longitudinal 
component to SASS rests on several 
premises. First, attaching a longitudinal 
study to SASS seems to satisfy most of the 
criteria I have set out for myself. It could 
measure all three of the types of student 
data deemed most worthwhile, while also 
satisfying the two criteria for sensible 
merger— producing some cost benefit, and 
engendering an analytical payoff. The 
payoff, however, is to the overall data 
collection effort of NCES and not 
necessarily to SASS data collection in 
particular. In fact, attaching a longitudinal 
study to SASS may have no payoff 
whatsoever for SASS but may indeed 
provide more burden to the already 
overworked SASS staff. Attaching 
aggregate longitudinal student data to 
SASS may be of more benefit to SASS 
itself— merging a new NELS and SASS 
provides the most benefit to NCES and 
indeed, to the whole educational policy 
community. 



Conclusion 

The years 1983-84 saw the release of two 
publications that would forever change the 
way that Americans looked at their 
elementary and secondary schools. Ernest 
Boyer’s High School. A Report on 
Secondary Education in America, 1983 
focused public attention on American high 
schools, a “troubled institution” with a 
confused mission and low standards. At 
about the same time the U.S. Department 
of Education released A Nation at Risk, 
which called attention to what was termed 
a “rising tide of mediocrity” in American 
schools. Due in part to the publicity these 



reports engendered, a decade of 
educational reform took hold in the 
American educational system. This 
“reform” was actually many reforms and 
debate over the consequences of these 
reforms continues today. NCES data help 
frame and focus this debate. 

In 1984, a cohort of students had just 
graduated (in 1982) from high school. 
Their experiences in the pre-reform era 
would serve as a base line to judge the 
impact of the coming reforms. The High 
School and Beyond study would record the 
experiences of this cohort of students. In 
1984, another cohort of students was in the 
fourth grade. These students would feel 
some of the immediate consequences of 
these reforms. Their experiences in high 
school, in postsecondary education, and in 
the transition to the world of work were 
captured in the experiences of the students 
in the National Educational Longitudinal 
Study of 1988. In 1984 (the year in which 
A Nation At Risk made its first impact), yet 
another cohort of children were born who 
are right now experiencing the full impact 
of the reforms of the last two decades. 

Most of this cohort are on track to 
graduate from high school in 2002. 

Unfortunately, current budget concerns 
cast doubt on whether NCES will be able 
to field an independent longitudinal study 
of this class of high school students. The 
cohort of students who will be included in 
the Early Childhood Longitudinal Study 
will not be graduating from high school 
until 2012. Missing the class of 2002 will 
result in a data gap of almost 20 years and 
will weaken our ability to measure the 
impact of the changes introduced into our 
elementary and secondary schools. Failing 
to capture the experiences of the high 
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school class bom at the very beginning of 
reform will be a serious gap in the nation’s 
knowledge about education. Linking a 
new longitudinal study with SASS may be 
the only way of effectively filling this data 

gap- 
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MAKING DATA RELEVANT FOR POLICY DISCUSSIONS: 
RECOMMENDATIONS FOR REDESIGNING THE SCHOOL 
ADMINISTRATOR QUESTIONNAIRE FOR THE 1998-99 SASS 1 

Henry Y. Zheng, Ohio State University 



Introduction 

As an integral part of the Schools and 
Staffing Survey (SASS) conducted by the 
National Center for Education Statistics 
(NCES), the School Administrator 
Questionnaire has been used to collect 
information from both public and private 
school administrators regarding their 
demographic characteristics, academic 
background, professional training, and 
attitudes toward school management issues 
(Davis & Sonnenberg, 1995). Thus far, 
three surveys were conducted separately in 
1987-1988, 1990-1991, and 1993-1994. 
These efforts have resulted in a large 
integrative database that can be used to 
present a comprehensive national profile of 
school administrators as a professional 
work-force. Research products based on 
this data source have already provided 
valuable information to education decision- 
makers on a number of important policy 
issues. 

For example, in their report to the 
National Leadership Network, Moorman 
and associates (1992) argue that there is a 
pervasive bias favoring white male 
principals over female and minority 
principals in America’s schools. They 
question whether female or minority 
principals may “inhabit a school different 
from their majority counterparts” and 
whether this difference may “hold 



significant implications for their tracking 
and on-the-job performance (p. 166).” 
Moorman and associates’ observation 
touches upon a sensitive issue that has long 
been debated within the education 
administration community. However, 
without the support of authoritative 
evidence, such an observation remains 
subjective and inconclusive. Fortunately, 
with the implementation of SASS, policy 
issues such as this can now be fully 
explored by tapping into the data resources 
collected through the school administrator 
survey. Within NCES, both Hammer and 
Rohr’s report (1994) on the disproportional 
distribution of male and female principals 
in America’s public and private schools 
and Rossi and Daugherty’s report (1995) 
on the types and locations of schools at 
which America’s minority principals work 
have rendered strong statistical evidence to 
support some of the arguments made by 
Moorman and associates (1992). 

The school administrator survey, together 
with other components of SASS, not only 
provides data for mapping the basic 
demographic and educational background 
of school administrators, it also offers 
opportunities for assessing principals’ 
attitudes toward school management issues 
such as the priorities of educational goals, 
seriousness of school problems, and the 
distribution of decision-making power in 
schools. As education reform continues to 
demand improvements and greater 
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accountability from our educational system 
to better prepare students for future 
challenges, it becomes evident that 
information regarding school principals 
will continue to be of great interest to 
education researchers and policymakers. 

As Odden (1995) points out, the 
decentralization of decision-making power 
from school boards to individual schools 
has placed school principals at the 
forefront of the current reform movement. 
It is therefore important that we have 
reliable and comprehensive information on 
the principalship in order to make sound 
judgments on school administrators’ roles 
and contributions to school improvement 
and student outcomes. 

NCES is currently in the process of 
reviewing the direction, purposes, and uses 
of SASS for the planned 1998-99 survey. 
This includes examining the current uses 
of its data, its relationships with other 
federally sponsored data collection efforts, 
and future national survey needs in 
accordance to changing policy priorities. 

As part of the review efforts, this paper 
will discuss the scope, uses, and possible 
changes of the school administrator 
component of SASS from a user’s 
perspective. In the following chapters, I 
will first present an overview of the school 
administrator questionnaire across three 
separate surveys. Next, an assessment of 
the current uses of the school administrator 
data will be given. Lastly, comments and 
suggestions for possible changes to the 
school administrator questionnaire will be 
offered. 



An Overview of the School 
Administrator Questionnaire of 
SASS 

The school administrator survey of SASS 
is the most comprehensive and the largest 
national study of school principals in this 
country, perhaps even in the world. The 
only study outside of NCES that can 
competently approach the depth and 
extensiveness of SASS is the survey of 
school administrators by Feistritzer and 
associates in 1987 for the National Center 
for Education Information. Feistritzer and 
associates’ study (1988) surveyed the basic 
demographic background of school 
administrators and their attitudes toward a 
number of school management issues. The 
study has a sample of more than five 
thousand elementary and secondary school 
administrators, including superintendents, 
public and private school principals and 
assistant principals. Compared to the 
school administrator survey of SASS, 
Feistritzer and associates’ study has a 
number of disadvantages. It is a one-shot 
study, hence it is limited by its inability to 
provide a longitudinal perspective on 
changes occurring within the school 
administrator workforce. It also lacks the 
broad scope that SASS has. Feistritzer and 
associates’ study only provide basic 
demographic information such as gender, 
age, education, and work experience. It 
does not have other equally important 
information such as principals’ 
professional training and the contextual 
environment of schools in which principals 
conduct their daily business. Moreover, 
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Feistritzer and associates’ survey did not 
offer user-friendly data resources to other 
researchers for further exploring the policy 
issues related to the principalship. 

Compared to Feistritzer and associates’ 
and other similar studies, data collected 
from the school administrator surveys of 
SASS have several distinct advantages. 

First of all, it has a large and 
comprehensive sample of principals from 
all varieties of schools. It includes not 
only principals from public schools of 
different sizes, locations, and levels, but 
also private schools of different group 
types and religious affiliations. The 1987- 
88 SASS has a sample size of 9,317 public 
schools and 3,513 private schools (NCES, 
1994) while the 1990-91 SASS has a 
sample size of 9,330 public schools and 
3,270 private schools (Kaufman & Huang, 
1993). Such a high degree of 
representation affords researchers the 
opportunity to conduct analysis down to 
the basic level of the stratification 
sampling structure. For example, there 
are even enough cases for comparing three 
different types of Jewish schools in the 
private school sample (McLaughlin et al., 



1995). Secondly, the school administrator 
survey is structurally integrated with other 
components of SASS. For every school 
included in the survey, its principal and a 
number of teachers within the same school 
would also be surveyed. The school’s file 
is also linked with the school district’s file. 
These inter-file linkages provide a high 
degree of flexibility to data users for 
incorporating relevant variables from other 
databases. For example, while the school 
survey provides contextual information 
regarding the schools in which principals 
fulfill their leadership roles, the teacher 
survey supplements additional information 
on how well principals perform such 
leadership roles (from teachers’ 
perspective). Moreover, the school 
administrator questionnaire has maintained 
a high level of consistency over the past 
three surveys that many of the core items 
remain unaltered. Such a consistency 
allows researchers to evaluate the changes 
over time in many areas of the 
principalship. Policymakers may use such 
data to assess the changes and progresses 
of the school administration workforce. 



Table One 

An Overview of Questionnaire Items of School Administrator Component of 
The Schools and Staffing Surveys at NCES 



Categories of Questions 


Specific Types of Questions Asked 


Number of 
Items 1 


1987-88 


1990-91 


1993-94 


Education and Training 


Degrees achieved and major fields of 
study 


25 


18 


29 2 




Professional Training 


4 


7 


6 


Professional Experience 


Teaching experience: years and 
assignment fields 


6 


4 


4 




Administrative experience: years 
and positions 


5 


5 


24 3 




Other job experience 


6 


6 


17 4 


Career plan 


Plan to remain as principal 


- 


2 


6 


Compensation 


Salary 


2 


2 


2 




Benefits 


10 


10 


10 


Demographics 


Gender, age, race 


4 


5 


5 


Job-related Activities 


Activities and hours spent 


11 


- 


- 


Perceptions 


Perceptions of school problems 


13 


22 


24 




Perceptions of influence on school 
matters 


9 


15 


39 5 




Perceptions on school educational 
goals 


- 


3 


3 


Questions about school’s 


Teacher evaluation 


1 


3 


- 


Teaching staff 


Teacher training 


1 


- 


- 




Teacher recruitment 


8 


- 


- 


Miscellaneous 


Data & telephone number 


3 


3 


4 


Total 




108 


105 


173 



1 Refers to total number of response items. A question may have multiple response items. 

2 BA/BS degree granting university and location were added. 

3 Grade level of previous principal positions and breaks in principal career were added. 

4 New position categories and years of experience were added. 

5 Private school version has 27 items. 
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In Table One, an overview of all survey 
items is presented for the school 
administrator questionnaires. While the 
total number of survey items for the first 
two surveys is about the same (108 and 
105 respectively), the number of items for 
the 1993-94 survey increases substantially 
to 173. Given the busy schedule of school 
administrators, it is reasonable to believe 
that this total number of question items has 
reached the critical length. Any increase 
in length will very likely cause a decline in 
survey return rate. Across the three 
questionnaires, there are eight general 
categories of questions: questions about the 
education and training of principals, their 
professional experience, their plan to retire 
or to remain as principals, their salary and 
benefits, their perceptions on a variety of 
school matters, their basic demographic 
background, their job-related activities, 
and their opinions of their teaching staff. 

Among these eight general categories of 
questions, items inquiring about principals’ 
education and training, professional 
experience, compensation, and basic 
demographic background remain consistent 
throughout all three surveys. These 
questions are the core items of the 
questionnaires. They are essential in 
tabulating the gender, age, and racial 
distributions of school principals and in 
presenting a basic profile of their 
educational background and professional 
preparation (including pre-service, in- 
service training and work experience). 

The availability of these data items enables 
the tracking of the dynamic changes in the 
basic characteristics of our nation’s 
education administration workforce. It 
also helps answer some of the fundamental 
questions about the reform of the principal 
workforce itself. For example, in an 



extensive study of the gender factor on 
principals’ career decisions, their job 
performance, their compensations, and 
their job satisfaction, Gross and Trask 
(1975) documented significant differences 
between male and female principals 
through personal interviews and mail 
surveys. Twenty years after Gross and 
Trask’s study, one wonders how our 
education system has been changed to 
achieve gender equity. To answer this or 
other related policy questions, these core 
data items from the school administrator 
surveys are particularly useful. 

Principals’ perceptions of school matters 
are another general category of question 
items. The school administrator 
questionnaires ask principals to indicate 
their perceptions regarding the seriousness 
of a variety of problems facing their 
schools, their perceptions over the 
distribution of decision-making influence at 
the school, and their ranking of important 
educational goals. 2 Over the years, these 
items regarding principals’ perceptions 
have proliferated. The items for mapping 
principals’ perceptions on school problems 
increased from 13 items in 1987-88 to 24 
items in 1993-94. Items regarding 
principals’ perceptions on the distribution 
of decision influence for school matters 
(also called “locus of control” items) 
increased from nine items in 1987-88 to 39 
items in 1993-94. 3 To a certain extent, 
such increases reflect efforts committed by 
the staff at NCES to make the SASS 
surveys more relevant to the policy debates 
over reform and restructuring in education 
administration. If we are to put more 
power into the hands of school principals 
and teachers to decide what is good for 
their schools and their students, we ought 
to know what they think about the merit of 



the reform and the impacts of its 
implementation. Understandably, most of 
these perceptual items could also be found 
in the teachers’ questionnaires. 

Question items about principals’ career 
plans, such as decisions to retire or to 
remain as principals, grow from none in 
1987-88 to six items in 1993-94. These 
items help shed light on the supply and 
demand situation of the school 
administrator workforce. Together with 
information about principals’ ages and 
career breaks, these data items can be used 
to assist the projection of demands for new 
principals. Judging by the fact that 
information regarding principals’ supply 
and demand situation is seriously lacking, 
the availability of these items can be quite 
valuable to those who have stakes in 
training and recruiting new school 
administrators. 

While most categories of questions in the 
school administrator questionnaires have 
experienced increases in question items 
over time, two categories of questions 
have been down-sized. Questions about 
principals’ job-related activities and 
questions about the recruitment and 
evaluation of teachers were actually 
eliminated from the 1993-94 survey. 

These changes may have been justified at 
the time the 1993-94 survey was planned. 
However, it is my opinion that these items 
should be restored but in different formats. 
Details will be discussed in Section IV of 
this paper. 



School Administrator Research 
Using SASS - An Assessment 

The three waves of SASS school 
administrator surveys have accumulated a 
large amount of data about school 
principals in the United States. Modern 
decision-support theory believes that data 
can be transformed into information only 
when it is used to assist decision-making 
(Rohrbaugh, 1986; Hammond & Arkes, 
1986). According to this view, the 
effectiveness of data-col lection efforts is 
ultimately judged by the quality of the data 
and how the data are used to inform policy 
decisions. In a report prepared by the 
Research Triangle Institute, Curtin and 
Fiore (1995) clearly indicate that the 
school administrator database from SASS 
is a very useful source of information for 
education decision-makers. In a sequence 
of topics, Curtin and Fiore demonstrate 
how the school administrator data can be 
used to inform policymakers and education 
researchers about the pluralistic 
transformation of the principalship in 
America’s schools, the changing 
qualifications and professional preparation 
of school principals, the new roles 
required for principals in managing 
schools, and the status of principal 
retention and turnover. 



ERIC 



72 



Table Two 

Samples of School Administrator Research Usnmg NCES Data 



Research Based on SASS 


Research Based on HS&B 


Author, Year, Title 


Type of 
Publication 


Author, Year, Title 


Type of Publication 


Hammer, C. and Rohr, C. (1993) 
Teaching, Administrative, and Other 
Work Experience of Public School 
Principals 


Issue brief 
NCES Publication 


Chubb, J. and Moe, T. 
(1985) Politics, Market, and 
the Organization of Schools 


Conference paper 
American Political 
Science Association 


Hammer, C. and Rohr, C. (1994) 
Public and Private School Principals? 
Are There Too Few Women? 


Issue brief 
NCES Publication 


Eberts, R. and Stone, J. 
(1988) Student Achievement 
in Public Schools: Do 
Principals Make a 
Difference? 


Journal article 
Economics of 
Education Review 


Rossi, R. and Daugherty, S. (1995) At 
Which Types of Schools Do Minority 
Principals Work? 


Issue brief 
NCES Publication 


Hannaway, J. and Talbert, J. 
(1991) Bringing Context into 
Effective Schools Research: 
Urban-Suburban Differences 


Research Paper 
OERI Publication 


Ingersoll, R. and Rossi, R. (1995) Who 
Influence Decision-making About 
School Curriculum: What Do Principals 
Say? 


Issue brief 
NCES Publication 


Goldring, E. and Rallis, S. 
(1993) Principals of 
Dynamic Schools: Taking 
Charge of Change 


Book 

Corwin Press 


Anderson, J. (1993) Who’s Runs the 
Schools? The Principal’s View 


Research Report 
OERI Publication 


Lee, V. et al. (1993) 
Teachers and Principals: 
Gender-related Perceptions 
of Leadership and Power in 
Secondary Schools 


Journal article 
Educational 
Evaluation and 
Policy Analysis 


Fowler, W. (1991) What Are the 
Characteristics of Principals Identified 
as Effective by Teachers? 


Conference paper 
AERA 


Brewer, D. (1993) 
Principals and Student 
Outcomes: Evidence from 
U.S. High Schools 


Journal article 
Economics of 
Education Review 


Haller, E. et al. (1994) Does Graduate 
Education in Educational 
Administration Improve America’s 
Schools? Another Look at Some 
National Data 


Conference paper 
AERA 






Ballou, D. and Podgursky, M. (1995) 
What Makes A Good Principal? How 
Teachers Assess the Performance of 
Principals 


Journal article 
Economics of 
Education Review 
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In Table Two, research products using the 
SASS principal database are listed together 
with research products using the High 
School and Beyond (HSB) principal 
database. HSB is a national longitudinal 
survey of high school sophomores and 
seniors conducted also by NCES. Students 
selected to participate in the study were 
administered a series of cognitive tests 
measuring their verbal and quantitative 
skills in 1980. Several follow-up surveys 
were conducted with sub-samples of the 
original sample population to determine 
changes in their test scores. In 1984, an 
“Administrator and Teacher Survey” 

(ATS) was added to the HSB study, with 
questionnaires administered to principals, 
guidance counselors, and teachers in about 
500 schools, or about half of the original 
number of sample schools (Chubb & Moe, 
1985). The added ATS was intended to 
study the organizational aspects of schools 
—schools’ relationships with parents and 
school boards, teachers’ perceptions of 
principals’ leadership roles, and principals’ 
perceptions of school environments and 
management practices. Many of the 
question items used in ATS were later 
incorporated into the teacher and principal 
components of SASS. It is therefore 
useful that in discussing the use of SASS 
data to study the principalship that we also 
discuss research products that are based on 
HSB data. 

Overall, it appears that research endeavors 
based on HSB data had enjoyed greater 
success in getting their products accepted 
by external publications (see Table Two). 
Research works using HSB data were 
accepted not only by academic journals 
and conferences, but also by a major book 
publisher that specializes in education- 
related topics. In contrast, only one of the 



principal-related research products based 
on SASS data was accepted for external 
publication. The relative success of HSB- 
based research products is partly due to 
the early inception of the HSB database. 
The fact that HSB data were collected 
almost four years earlier than SASS data 
gave HSB data-users much more time to 
get familiarized and to work with the data. 
Another reason that may explain the 
relative success of HSB-based studies is 
that principal and teacher questionnaires 
were added to the original HSB survey of 
students’ cognitive abilities with a clear 
intention of linking principals’ 
performance to student outcomes. This 
addition enabled the merging of the 
demographic background and personal 
perceptions of principals and teachers with 
students’ test scores. This merging 
provides the convenience and opportunities 
for exploring the connections between 
principal-related variables and student 
outcomes. 

In a study of principals’ influence on 
student outcomes based on HSB data, 
Brewer (1993) used the change in student 
test scores between sophomore year (1980) 
and senior year (1982) as a dependent 
variable to measure the influence of 
principals on student outcomes. His study 
reveals that principals can influence 
student outcomes through the setting of 
academic goals for students, through the 
screening of new teachers, and through 
their decisions on instructional methods. 
This study, together with the study by 
Eberts and Stone (1988), is one of the very 
small number of empirical studies on 
principals’ influences on student outcomes 
that were reportedly backed by direct 
statistical evidence. Brewer claims that his 
findings render supports to the “effective 
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schools” argument that principals can 
make a difference in student’s outcomes 
(Brewer, 1993). 

Despite the optimism, research findings 
based on the linking of ATS data with 
HSB data are not without their perils. First 
of all, the connection between principal 
behaviors and student performance is 
indirect at best (Glasman & Heck, 1992; 
Kleine-Kracht, 1993). Principals do not 
interact with students directly and their 
influences on student achievements are 
muted by other more powerful factors, 
such as the quality of teachers, the degree 
of parental involvement, and students’ Own 
motivations, just to name a few. It is 
difficult to imagine that these variables can 
be controlled in the analysis. Therefore, 
caution needs to be exercised in making 
direct inference from principals’ 
performance to student outcomes. 
Furthermore, the use of students’ cognitive 
test scores as the sole basis for assessing 
student outcome is too simplistic. Student 
outcome is a multi-dimensional construct. 
To reduce such a broad concept into a 
single dimension solely based on test 
scores undermines the moral and social 
values of education, not to mention that 
test-based criteria can also be quite 
complicated. The way Brewer (1993) used 
the data also led to some unsettling 
questions about the reliability of his 
findings. For example, the dependent 
variable he used is the change in student 
test scores between 1980 and 1982, but the 
independent variables are from principal 
and teacher surveys conducted in 1984. In 
light of the time differences, we need to 
ask: Are those principals surveyed in 1984 
the same principals in those schools 
between 1980 to 1982? Did teachers and 
principals have the same perceptions in 



1984 as in 1980 or 1982? Moreover, 
there were drop-outs in the test population 
between 1980 and 1982. It may be 
reasonable to speculate that these drop-outs 
are probably among the students who did 
not perform well in the first cognitive test. 
Did this selection bias have influence on 
the internal validity of the research 
findings? 

In comparing the ATS of HSB and the 
principal and teacher components of SASS, 
Ingersoll (1995) points out quite clearly 
the differences between the two databases. 
ATS was developed specifically to 
facilitate the investigation of relationships 
between school staffing characteristics and 
student outcomes (Chubb & Moe, 1985) 
while SASS is intended to provide a 
comprehensive assessment of schools and 
staffing conditions in the United States. 
ATS has a usable sample of about 350 
secondary schools while SASS has more 
than 12,000 sample schools of different 
levels. ATS has a limited generalizability 
of schools due to its small sample size 
while SASS supports national estimates by 
any number of different school 
characteristics, including sector, level, 
state, urbanicity, and school size. SASS is 
also more accurate in distinguishing 
different types of private schools. For 
example, SASS separates private schools 
into sub-types grouped by their religious 
orientations and group affiliations. 

The comprehensiveness of the SASS data 
and the availability of three waves of 
surveys have provided ample opportunities 
for conducting in-depth analysis on a 
number of key education policy issues. 
With direct relevance to education 
administration research, we may use the 
data to study the changing demographic 
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characteristics of the education 
administration workforce and how these 
demographic changes are associated with 
changes in salary and compensations. 
Policy studies such as these can answer 
questions on whether economic equity for 
women and minority principals has been 
improved as their shares in the 
administration workforce expanded. We 
may also pursue studies to understand the 
perceptual differences between principals 
and teachers regarding the decision-making 
structures in schools and how such 
differences in perceptions are conditioned 
by their educational and demographic 
background. Such studies may help 
explain the trends and patterns of 
decentralization and the locus of control in 
school management. Or, we may want to 
find out whether the effectiveness of 
principal leadership is constrained by the 
organizational settings or the socio- 
economic conditions of schools. The 
study of principal leadership using SASS 
data, even without the direct linkage to 
student outcomes, can still shed light on 
how schools can be more effectively 
governed and improved. 

Over the past several years, there has been 
a number of principal-related research 
papers that based their findings on the 
analysis of the SASS data. In addition to 
the issue briefs and research reports 
published by the National Center for 
Education Statistics, there are two 
academic conference papers and one 
published journal article that employed the 
SASS data to address issues surrounding 
the principalship and school 
administration. For example, Fowler 
(1991) used teachers’ perceptions of 
principals’ leadership behaviors to create 
an index of perceived principal 



effectiveness. His findings indicate that 
principal effectiveness is a complex and 
multi-dimensional construct and a 
principal’s ability in leading the school 
effectively is influenced by a number of 
factors, including the principal’s age, 
gender, teaching experience or the school’s 
location, size, and level. Similarly, the 
study conducted by Haller, Brent, 
McNamara, and Rufus (1994) also used 
teachers’ perceptions of principals to 
create indices of school leadership, but 
Haller and associates’ interest is to find 
out how graduate training in educational 
administration would help improve 
principals’ leadership effectiveness. The 
findings from Haller and associates’ study 
lead to a disturbing yet tantalizing 
conclusion: graduate training in education 
administration does not have significant 
influence on the attributes that characterize 
effective principals. Judging by the fact 
that most states require a master’s degree 
or even a doctoral degree in Education 
Administration as a prerequisite for 
principal licensure, this finding posts 
serious questions on the validity of such a 
requirement to education policymakers. 
Recently, the Los Angeles Unified School 
District decided to waive this and other 
mandated requirements for hiring new 
principals, 4 hence setting an example for 
challenging the status quo of school 
administration licensing practices. This 
bold reform effort is clearly supported by 
what Haller and associates discovered from 
analyzing the SASS data. 

In the only externally published research 
paper that employed SASS data to address 
principalship issues, Ballou and Podgursky 
(1995) used the 1987-88 SASS data to 
evaluate the influence of principals’ 
educational credentials and professional 
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experience on teachers’ assessment of 
principals’ leadership practices. Ballou 
and Podgursky find little statistical 
evidence to support the recent proposals 
for enhancing the “professionalism” of the 
principal workforce by requiring more 
advanced degrees and additional 
administrative training. They argue that 
principal licensing requirements in the 
states may undervalue those attributes that 
characterize good school leaders. For 
example, principals with more teaching 
experience are generally rated higher by 
teachers. However, most states only 
require new principals to have a few years 
of teaching experience. Ballou and 
Podgursky’s study obviously raise serious 
questions for policymakers to ponder. 

Despite the relatively small number of 
studies using SASS to examine the roles 
and practices of school principals, those 
that had been completed have provided 
many interesting and fresh insights on 
policy issues related to school 
administration. Given the great potential 
of SASS as a comprehensive national 
database on schools and their staffing 
patterns, it is imperative that more studies 
be done to take advantage of the depth and 
richness of the database. In order to 
encourage more researchers to use the 
SASS data to study education policy 
issues, NCES must play a more active role 
in facilitating the awareness of and access 
to the dataset. For example, a brochure 
describing the database together with 
suggested research questions can be 
distributed to directors of graduate studies 
at universities to encourage doctoral 
students to utilize the database for writing 
their dissertations. The electronic 
codebook system now available with the 
SASS CD-ROMs should be transformed 



into a Windows-based system to enhance 
its user-friendliness. 5 This change is 
imperative now given the fact that new 
Windows operating systems such as 
Windows 95 or Windows NT are phasing 
out DOS-based software. Alternatively, 
SASS can create its own world-wide web 
(WWW) home-page to provide online, 24- 
hour access to the database. 6 

Efforts to promote the awareness and 
access of the SASS database will provide 
the necessary conditions for wider use of 
the data resources for education research. 
However, in order to facilitate the use of 
the data to conduct policy analysis, 
modifications must be made to the content 
of the questionnaire so that it is more 
relevant to current policy debates. In the 
following section, I will discuss my 
personal view on how to improve the 
principal questionnaire. 

Improving the School 
Administrator Questionnaire: 

Some Suggestions 

In a discussion of methods for assessing 
the effectiveness of public funded research 
organizations, Altschuld and Zheng (1995) 
believe that a stakeholder-based approach 
is more useful than a goal-based approach 
in assessing the performance of research 
organizations. This is because research 
organizations usually have broad and 
general goals and their organizational 
outputs are mostly intellectual products 
and services that cannot be measured 
meaningfully in tangible monetary or 
numeric terms. Performance of research 
organizations thus are better gauged from 
their customers’ perspective. In the case 
of SASS, its customers would include 
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education researchers, planners and 
policymakers at federal, state, and local 
governments, and individuals and 
organizations who have interest in schools 
and school staffing issues. 

In order to improve the relevance and 
usefulness of the data products of SASS, it 
is logical that we find out what the 
customers’ current and future needs are. 

To this end, I decided to obtain some first- 
hand knowledge by conducting a small 
survey via the Internet discussion group 
“AERA-A 7 ” hosted by Arizona State 
University. 8 In my e-mail survey, I 
asked, “what is the most important policy 
research issue for education administration 
in the next 10-15 years?” Of the 18 
answers with direct relevance to the 
principalship, 28 percent of the answers 
(n=5) indicated that principals’ roles and 
contributions in school decentralization and 
restructuring should be the most important 
research issue; 33 percent of the answers 
(n=6) stated that principals’ accountability 
to school outcomes should be the key 
issue. Specifically, one respondent wanted 
to know how principals can be evaluated 
fairly through demonstrated added value to 
the learning process of students; 22 
percent of the answers (n=4) identified the 
working conditions and career decisions of 
principals as the major research concern. 
Lastly, 16 percent of the responses (n=3) 
rated principals’ attitudes and handling of 
new information technologies in classroom 
teaching and learning as the most 
important issue. 

This survey may be too small to collect the 
opinions of the broad research community 
of education administration; however, a 
review of the literature shows that the 
research issues raised by these respondents 



actually echo with some of the 
community’s prevalent views. Murphy 
and Beck (1994) believe that principals’ 
roles and responsibilities must be clarified 
at a time when schools are forced to 
transform and restructure. Despite the 
increased importance of their jobs, 
principals themselves have been silent and 
passive in defining their roles in school 
leadership. The clear definition of 
principals’ roles will help principals 
understand how to meet the demands for 
accountability, how to adapt to the 
changing social fabric of schools, and how 
to make schools meet the needs of a post- 
industrial world. In a widely cited paper, 
Re-Thinking School Leadership: An Agenda 
for Research and Reform, Bolman, 

Johnson, Murphy, and Weiss (1990) of the 
Harvard Graduate School of Education 
make similar observations. They believe 
that school leaders, under the constraints 
of changing student demographics, fiscal 
difficulties, and greater societal 
expectations, must learn to cultivate 
constituent supports, including support 
from parents, teachers, and school boards. 
School leaders should adapt to new 
management reality and to establish 
mutually dependent and cooperative 
relationships with teachers. School leaders 
should also understand how the technology 
of instruction can facilitate the delivery of 
knowledge from teachers to students. 

Most of the recent studies on principals’ 
roles in school restructuring and reform 
are built on the framework of the 
“effective school” movement (Murphy & 
Louis, 1994; Hallingert & Leithwood, 
1994). Despite the marginal variations in 
their views, these studies in general agree 
with Murphy’s (1994) argument that 
school restructuring produced a nearly 
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overwhelming workload for principals, 
demanded that they work both harder and 
smarter, and created considerable work- 
related stress. In order to survive these 
new challenges, principals must adapt to 
roles changes in several important areas: 
delegating more decision-making power to 
teachers and other support staff and 
promoting a collaborative relationship with 
them; enabling and supporting teacher 
success through more constructive 
approaches in professional developments; 
managing a constellation of change efforts, 
including the direct involvement in 
instructional practice improvement; and 
extending the school boundary through 
active community marketing efforts to 
cultivate parental and public support 
(Bookbinder, 1991; Elmore, 1995; Odden, 
1995). 

Judging from the above discussions on the 
important policy research issues for 
education administration in the years to 
come, it is quite clear that principals’ new 
roles in leading schools at a time of 
change and uncertainty is of major concern 
to education researchers and policymakers. 
If SASS is to contribute more to these 
policy discussions, its survey 
questionnaires must be updated from its 
current format to become more relevant to 
the needs of the educational policy 
research and decision-making 
communities. In light of these discussions 
and the objectives of SASS, I would like 
to make the following suggestions for 
bringing the current SASS school 
administrator questionnaire (1993-94 
version) to tie more closely to the current 
policy debates: 



Core Items 

The core items of the questionnaire, that 
is, questions about principals’ education 
and professional training, work 
experience, compensations, and 
demographic background should be 
retained for all surveys. These items are 
deemed essential, for they provide the 
necessary conditions for constructing the 
basic profile of the school administrator 
workforce. Policy issues may change and 
research emphases may shift over time, 
but the need for understanding the basic 
characteristics and working conditions of 
school principals is continuous. 

Researchers, policymakers, and other 
concerned parties need these core data 
items to track the dynamic changes in our 
nation’s education administration 
workforce. Policy research based on the 
analysis of perceptions, school outcomes, 
or community opinions also need to 
consider the contextual constraints of the 
principalship. Therefore, it is imperative 
that these items be retained for all surveys. 

Nevertheless, some of the questions in the 
core items are too elaborate and some of 
them are too simple. Changes should be 
made to balance the two extremes. 
Specifically, questions about principals’ 
education background may be too 
elaborate. For example, it may not be 
necessary to ask whether a principal has an 
associate’s degree and in what field. Given 
the fact that the majority of principals has 
at least a master’s degree and nearly 
everyone has a bachelor’s degree, what is 
the value of knowing whether one has an 
associate’s degree? 9 It is also not 
necessary to ask about the location and 
name of the college from which they 
earned their college degrees. There is no 
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written rule that a principal must graduate 
from an Ivy League school or a top-tier 
public university. Knowing the schools 
from which they graduated will not 
contribute significantly to the 
understanding of the policy issues 
surrounding the principalship. 

Question #16, "were you the principal of 
this school in the Spring of 1991?" is 
redundant since the question right after it 
asks the respondents, "prior to this school 
year, how many years were you employed 
as the principal in this school?" If the 
answer is greater than two, by logic, the 
respondent would most probably be the 
principal of this school in the Spring of 
1991. Question #18b asks, "in what 
grade levels were the students in the 
school in which you last served as 
principal?" There are 15 choices, ranging 
from pre-kindergarten to grade 12. Each 
choice takes up one data space. In order 
to increase the efficiency of the answer 
format, it would be better that a smaller 
range of choices was used in this case. 

For example, instead of listing all possible 
grade levels, the question can simply have 
several general categories of answers (i.e., 
elementary, secondary, elementary and 
secondary combined, and others). 

In addition to formal education, principals 
usually receive other types of training to 
prepare themselves for the job as a school 
leader. Exactly what kinds of training did 
they receive and how effective was that 
training in helping them prepare for the 
challenges ahead? We have no way to 
know, for the questions on professional 
training are simply too vague (only "yes" 
or "no" answers are available). Since pre- 
service and in-service training are very 
important parts of the principal 



certification and preparation process, it is 
therefore necessary that we ask more 
questions about them. In a recent article 
on principal training programs, Bjork and 
Ginsberg (1995) criticize the conventional 
approach to principal training as too 
abstract from reality to be of real-life use 
in practice. They believe that principal 
training programs in the United States are 
in need of a paradigm realignment, that is, 
fundamental changes that will force the 
entire field to shift from academic-oriented 
to practice-oriented training. These 
changes may include sending university 
faculty members to schools to gain clinical 
experience or release principals from 
school-level duties to pursue full-time 
training that will integrate their school 
administrative experience with theories. 

Observations made by researchers (Bjork 
& Ginsberg, 1995; Ballou & Podgursky, 
1995; Haller et al., 1994) regarding 
principal training programs clearly indicate 
that much needs to be learned about the 
content and impact of these programs. In 
order to facilitate the policy changes for 
improving these training programs, the 
principal survey of SASS should expand 
the number of questions in this area. In 
addition to asking whether a respondent 
had participated in a training program, we 
should also ask how long the training 
lasted, how frequently he/she received the 
training, and how satisfied was he/she with 
the outcome of the training. For 
example, we can ask: 

Have you participated in any in-service 
training in evaluation and supervision? 

Yes No 
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If yes, how many times have you had this 
type of training during your entire 
principal career? 

Times (give a number) 

To what extent did this training help you 
become a more effective principal? 

Extremely helpful Very helpful 

Somewhat helpful Not helpful 

Waste of time 



Another area that should be modified is the 
questions about service breaks in a 
principal’s career. Instead of asking 
whether the breaks were due to layoff or a 
reduction-in-force, multiple choices should 
be offered. For example, reasons for 
breaks in services could be: layoff, 
organizational restructuring, educational 
leave, personal/sick leave, military leave, 
and others. Through multiple choices, we 
may be able to know more about the 
reasons why principals have to leave their 
jobs. Additionally, we should ask how 
long it took them to find a new principal 
position if they were unable to return to 
their original positions. This information 
would shed light on the demand and 
supply situation for school principals. 

Principals’ Jobs and Responsibilities 

In the 1987-88 SASS survey, principals 
were asked to give their best estimates of 
the number of hours they spent on several 
categories of school -related activities, i.e., 
teacher supervision or curriculum 
management. In the two SASS surveys 
that followed, this entire set of questions 
was eliminated. Through my 
conversations with the staff at NCES, I got 
the impression that it was eliminated due 
to questions about the reliability of 
principals’ self-reported numbers and a 



significantly high percentage of missing 
answers (one possibility is that principals 
did not keep good track of exactly how 
they spent their time and were unwilling to 
venture a guess). If these problems are 
real, they would definitely create problems 
to the reliability of these data items, hence 
their elimination is the sensible thing to 
do. 

However, the elimination of these items 
created a regrettable void in the principal 
questionnaire of those items about the jobs 
and responsibilities that principals 
performed. It is regrettable because 
modern principalship is such a complex 
enterprise that most outsiders really cannot 
comprehend the extent to which these jobs 
and responsibilities burden school 
principals. 10 If we don’t ask these 
questions in our surveys, we miss the best 
opportunity to understand how school 
principals perceive and perform their 
responsibilities. The need to know more 
about the jobs principals perform is greater 
than ever. As my previous discussions on 
the key policy research issues indicate, 
principals presently are given more 
responsibilities as education reform, 
political changes and technological 
improvements have shifted more decision- 
making power from districts to schools. 
With the increase in responsibilities, calls 
are heard with increasing frequency for 
greater accountability and more extensive 
performance review for principals (Kirst, 
1990). 

Paradoxically, the increase of 
responsibilities does not come with better 
understanding of principals’ jobs and their 
abilities to perform those jobs. 

Gottfredson and Hybl (1987) provide a 
very good observation on this paradoxical 



situation. They believe that much of the 
demands for principals’ increasing 
accountability to school effectiveness are 
“based on very limited knowledge of what 
principals actually do and which aspects of 
the job are most important and most 
burdensome. Furthermore, although much 
writing and advice on the principalship is 
generic, the role of the principal may 
differ according to the kind of school the 
principal leads. Most principals must 
learn the ropes on the job with limited 
support and guidance. Many schools do 
not have a clear written job description to 
spell out what is expected of the principal 
(p. 1).” Clearly, to know more about the 
jobs and responsibilities of principals 
should be an important goal of a national 
survey of school principals. If data are 
collected for making informed policy 
decisions, then, data about principals’ 



roles and responsibilities are obviously the 
type of data that have a very high degree 
of policy relevance to decision-making. A 
major study of school administrators such 
as SASS simply cannot ignore this critical 
aspect of the principalship. 

In order to include question items on 
principals’ jobs and responsibilities in the 
questionnaire and not to repeat the pitfalls 
of the 1987-88 SASS survey, the questions 
must be framed differently. Instead of 
asking principals to provide estimates on 
time usage, we may ask them to rank the 
importance of a number of jobs related to 
their management responsibilities and how 
they actually allocate time to accomplish 
those jobs. For example, we can ask 
principals questions in the following 
format: 



Among the following school-related activities, please provide us your ratings of their importance to your job as a school leader and the time 
you spent on them given your current workload: 



Activities 

Sit in a classroom to observe 
teachers’ instruction 


Importance 

0- Not a part of my job 

1- Not important 

2- Somewhat important 

3- Moderately important 

4- Very important 


Time Spent 

0- None 

1 - Little 

2- Occasionally 

3- Frequently 

4- Extensively (a major part of my job) 


Talk to parents about their 
children’s school problems 


0- Not a part of my job 

1- Not important 

2- Somewhat important 

3- Moderately important 

4- Very important 


0- None 

1 - Little 

2- Occasionally 

3- Frequently 

4- Extensively (a major part of my job) 


Take actions to ensure enough 
computer & telecommunication 
equipment for students 11 


0- Not a part of my job 

1- Not important 

2- Somewhat important 

3- Moderately important 

4- Very important 


0- None 

1 - Little 

2- Occasionally 

3- Frequently 

4- Extensively (a major part of my job) 



By presenting questions in this format, we 
can avoid asking principals to pinpoint the 
exact number of hours they spent on each 
activity and to preserve the opportunity to 
obtain valuable information about their 



jobs and responsibilities. The scales for 
the answers can be fine-tuned to better 
capture the importance and the time spent 
on each activity. Further studies are also 
needed to find out what activities should 
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be included in the list. But it is certain 
that these activities should represent those 
jobs and roles typically performed by 
principals in their capacities as the 
cultural, managerial, instructional, moral, 
and strategic leaders (Leithwood & Duke, 
1993). Not all jobs and responsibilities 
are viewed as equally important to 
principals. Some responsibilities may be 
viewed as less important but would 
consume more of their time. And some 
responsibilities may be viewed as 
important but they are unable to devote 
more time to do. If we need to know how 
principals can effectively improve their 
schools, we at least should know 
something about how they perceive their 
roles and responsibilities and how much 
time they have to spend on each activity. 

Principals’ Perceptions of Their Teaching 
Staff 



In the 1987-88 SASS survey, principals 
were asked about the availability of formal 
evaluation systems for teachers. In the 
1990-91 SASS survey, this set of questions 
was changed to solicit principals’ 
perceptions on the quality of their teaching 
staff. However, in the 1993-94 SASS, 
these questions were removed completely. 
Given the usefulness of these items and the 
fact that there are only 2-3 items for this 
set of questions, its removal is also quite 
regrettable. Slater and Teddlie (1992) 
believes that an effective school must 
possess three key components: teacher 
preparedness, student readiness, and 
administrative appropriateness. These 
three components must be integrated into 
an unbroken chain of actions in order to 
generate better school outcomes. 

Principals may have influence on 
improving students’ achievements, but 



such influence to a great extent have to 
rely on teachers’ performance as a medium 
to deliver the effects. 

Although the teacher component of SASS 
has already provided large amount of data 
on teachers’ quality, they are from 
teachers’ own perspective. The addition 
of a few items in the principals’ 
questionnaire will give us an additional 
perspective on teachers’ quality. Since we 
have asked teachers to evaluate the 
performance of their principals, we should 
also ask principals to tell us how they feel 
about their teachers. Current efforts to 
reform our schools call for principals to 
work more closely with their teaching staff 
to improve student outcomes. In order to 
assess how the collaborative relationship 
between principals and teachers can 
flourish and how such a collaboration 
affect the overall effectiveness of the 
school, it is useful that we gain an 
understanding of both principals’ and 
teachers’ perceptions of the other party. 

Principals’ Perceptions on School Matters 

In all three waves of SASS surveys, 
principals and teachers were asked to 
reveal their perceptions on the seriousness 
of a range of school problems, issues 
related to decision influence (locus of 
control) on school matters, and the 
importance of a number of educational 
goals. In the 1993-94 survey, the total 
number of items for the perception of 
school problems is 24 for both principal 
and teacher questionnaires. In addition, 
the public school principal questionnaire 
has 39 “locus of control” items while the 
private school principal questionnaire has 
27. Since SASS already has three surveys, 
in retrospect, judging by the frequency of 
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the items being used and the consideration 
for reducing some questions in order to 
make room for new items, I would like to 
suggest that the entire section on 
principals’ perceptions of school problems 
be removed from the next survey and the 
items regarding “locus of control” be 
retained. 

I believe that the central objective for 
knowing principals’ and teachers’ 
perceptions on school problems such as 
student tardiness and student drug use is to 
provide policymakers and researchers data 
on how school administrators and teachers 
feel about the problems facing schools. 
Information about these perceptions can 
alert the public and decision-makers to 
give higher priorities to support principals 
and teachers to solve these problems. 

Since teachers interact directly with 
students and have first-hand knowledge of 
students’ conditions inside and outside of 
the classroom, we would assume that they 
at least have equally valuable comments on 
school problems. Teachers’ perceptions 
may be different from principals’. But for 
the purpose of understanding school 
problems facing students inside and outside 
of classrooms, teachers’ perceptions should 
be sufficient to help inform us of the 
seriousness of those problems. 

The “locus of control” items are a 
different matter, for the central objective 
here is to find out the perceptions on the 
distribution of decision influence among a 
number of people. Knowing the 
differences between principals’ and 
teachers’ perceptions help researchers and 
policymakers understand the decision- 
making and organizational structures in 
schools. Such an understanding in turn 
can help evaluate current efforts in 



restructuring the organizational 
arrangements for school governance. 

Since teachers and principals are 
increasingly charged with more authority 
in determining the curriculum, personnel, 
and discipline policies of schools, it is 
naturally necessary that both parties’ 
perspectives be considered. 

Issues regarding the organizational 
arrangements for power sharing in schools 
are sensitive yet important. Despite the 
obvious reasons for principals to work 
closely with teachers to achieve school 
outcomes, there are many problems that 
may lead this collaborative relationship to 
falter. Wooster (1991) believed that part 
of the problem could be attributed to each 
party’s perception of their domain of 
influence. For example, teachers may feel 
that they should have the most say in 
instructional matters. Therefore, when a 
principal visits a teacher’s classroom to 
observe instructional practice and make 
comments on possible improvements, the 
teacher may have the impression that the 
principal is interfering with his/her right to 
teach and is imposing an administrator’s 
view on the teacher who may be a better 
expert on the subject. Other issues that 
can be explored with these “locus of 
control” items are the differences between 
private and public school principals and 
between private and public school 
teachers. In a survey of Catholic teachers, 
Kushner and Helbling (1995) point out that 
private school teachers tend to agree more 
with their principals on school 
management issues and that such 
agreements are mostly based on mutual 
trust, while such trust and agreement are 
much weaker among public school teachers 
toward their principals. How true is this 
observation? Does this difference 
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contribute to the cultural differences 
between public and private schools? We 
can find out some answers by comparing 
the perceptions between public and private 
school principals and teachers. 

Principal Preparation and Licensure 

As I mentioned earlier in this paper, 
information regarding principals’ pre- 
service and in-service training and 
preparation are not detailed enough to 
provide good estimates on the impacts of 
these training and preparation programs on 
principals’ leadership effectiveness. In 
addition to my previous suggestions for 
expanding these questions, I would also 
like to see the inclusion of several 
questions regarding principals’ licensure in 
the next SASS survey. Almost all states 
require principals to possess a legitimate 
school administration license and to renew 
the license after a period of time in 
service. Reading through the job 
advertisements for principals, one cannot 
help but notice that a principal’s license is 
always one of the most important 
prerequisites for the job. Given such an 
emphasis on principal licensure, one has to 
wonder whether such a requirement has 
been helpful in keeping the principal 
workforce to a higher standard; or did the 
licensing process keep some of the 
brightest minds from the teacher workforce 
or other professions away from this 
important and challenging field of 
leadership? 

Despite the relevance of principal licensure 
practice to the formation of the principal 
workforce, information regarding this 
practice is scarcely available. It is 
therefore useful that in at least one of the 
principal surveys of SASS that we can 



devote some attention to this issue. In the 
questionnaire, we may ask principals when 
they obtained their first principal license 
and at what level, how many renewals did 
they have after the first license, whether 
they needed to apply for a new license 
when they transferred from an out-of-state 
administrative position, or whether the 
licensing process helped them become 
more effective school leaders. 



Conclusion 

In conclusion, I would like to reiterate the 
importance and usefulness of the principal 
surveys of SASS in contributing to the 
understanding of the characteristics and 
conditions of the school administrator 
workforce in America. As the most 
comprehensive study of school principals 
currently available, the principal survey of 
SASS has provided valuable data for 
exploring various important policy issues 
regarding the basic characteristics of 
school principals in the United States, 
including their education background, 
professional training, work experience, 
salary and compensation, and their 
perceptions on a range of school 
management matters. 

In order to further extend the principal 
survey’s utility in educational policy 
debates, I have suggested above a number 
of changes to the principal survey 
questionnaire. These suggestions include: 
keep the core items consistent throughout 
all survey efforts but simplify those items 
that are overly elaborate; expand the items 
on principals’ in-service and pre-service 
training programs and solicit principals’ 
level of satisfaction with those programs; 
request that principals rank the relative 



importance of a number of school 
activities as related to their role as school 
leaders and ask how they allocate their 
time for those activities; remove 
principals’ perceptions of school problems 
to make room for new items; retain the 
“locus of control” items; and include some 
questions in the next survey regarding 
principal licensure procedures. 

These suggestions are based on my 
understanding of the major policy research 
issues for education administration in the 
near future. The changes I suggested do 
not include possible items to evaluate how 
principals can create “added value” 
directly to student achievement as 
suggested by some scholars in my e-mail 
survey. It is not the objective of the 
Schools and Staffing Survey to assess the 
immediate impact of principals on student 
outcomes. It is also my contention that 
principals’ influence on students’ learning 
are indirect as long as teachers are the 
ones who teach in the classroom. Given 
these constraints, it is natural that 
principals’ demographic characteristics, 
their educational and professional 
backgrounds, their perceptions of school 
management issues, their perceptions of 
their teaching staff, and their economic 
status should be the major concerns of a 
national survey of school principals. 
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http://www.ed.gov/NCES. The Electronic Codebook for SASS 1987-88 is currently available. 

7. AERA-A is a group organized by the Education Administration section of the American Educational 
Research Association and has members from the research, practice, and policymaking communities of 
education administration. 

8. Records of the survey are archived at Internet site “magnus.acs.ohio-state.edu”. Electronic copies are 
available upon request by sending an e-mail to yzheng@magnus.acs.ohio-state.edu. 

9. In the 1993-94 SASS, only 4.9 percent of principals indicated that they had an associate’s degree. 

10. One of the reasons that the principal questionnaire is much shorter than the teacher questionnaire is the 
consideration that principals are under greater time and work pressures. 

1 1 . This question partially addresses one of the policy issues raised by several respondents in my e-mail survey 
about principals’ support for new technologies in schools. 
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MEASURES OF INSERVICE PROFESSIONAL DEVELOPMENT : 
SUGGESTED ITEMS FOR THE 1998-1999 SCHOOLS 
AND STAFFING SURVEY 

Dorothy M. Gilford 



PART I 

Inservice Professional Development 
in the United States 

What information do we need about 
inservice professional development? 
Without attempting to be comprehensive, a 
number of questions immediately come to 
mind: 

(1) How is inservice professional 
development (IPD) planned and 
coordinated? 

(2) Is the school environment 
supportive of IPD? 

(3) What is the range of programmatic 
approaches? 

(4) What are teachers doing to 
strengthen their practice— what is 
the format, location, length, and 
content of their IPD programs? 1 

(5) What are the teachers’ perceptions 
of the effectiveness of the IPD 
programs in which they participate? 

(6) How effective are the IPD 
programs in improving teaching 
and in enhancing students’ 
learning? 



(7) How much effort are teachers 
expending on IPD in terms of time 
and their own money? 

(8) How much support/encouragement 
are they receiving for IPD in the 
form of incentives, financial 
support, and time for IPD? 

(9) How prevalent are induction 
programs for beginning teachers, 
what areas are addressed in these 
programs, who provides support, 
how effective are the programs? 

(10) What is the level of public sector 
investment for IPD and what is it 
purchasing? 

(11) Are there better ways to invest 
these resources? 

(12) How are the characteristics of IPD 

changing over time? 

(13) What can we learn from IPD in 
other countries (especially those 
whose students do well in 
international assessments) that 
might help improve IPD in the 
United States? 

(14) What changes are needed in 
inservice professional development 
to meet the challenges of the 
current systemic reform movement 
in the United States? 
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Information relevant to most of these 
questions can be obtained from the 1998- 
1999 SASS. Exceptions are questions 6 
and 10. Smaller, sharply focused studies 
would be more appropriate for studying 
the effectiveness of IPD programs in 
improving teaching and in enhancing 
students’ learning. Nor is it feasible to 
estimate the level of public sector 
investment for IPD from a survey of 
schools and staff, since information would 
be needed from many sources other than 
the schools, e.g., federal, state, and 
district agencies. The subject of question 
10, induction programs, is not addressed 
in this paper, but will be the subject of a 
later paper. 

To respond to the other questions, it seems 
appropriate at this early stage of the 
development of the 1998-1999 SASS to 
suggest some possible items for SASS that 
would provide information related to the 
questions. The items will not be defined 
in great detail, since the purpose is to 
stimulate discussion about the value and 
feasibility of including the items in SASS- 
an approach that is consistent with the 
purpose of this paper which is to 
recommend items about IPD for possible 
inclusion in the 1998-1999 SASS. 
Throughout the paper, when an item is 
suggested that would provide information 
related to the questions, a footnote 
indicates the related questions. 

The first part of the paper is limited to 
IPD in the United States, while Part II 
discusses the value of international 
comparisons of IPD generally and in 
particular for use of computers and 
advanced telecommunications equipment. 
The focus on IPD for computers in this 
paper is to provide ample time for 



consideration of a large potential addition 
to SASS on this topic in 1998-1999. 

Part I starts by considering various 
definitions of IPD and its evolution-which 
is reflected in the definitions and the many 
types of IPD programs they encompass. 
Turning to the design of SASS items, it is 
noted that the large number of types of 
IPD programs calls for a framework to 
organize information collection and 
compilation. Such a framework is 
proposed and is used in the development 
of items related to the prevalence of IPD 
types. Since data from the 1998-1999 
SASS will become available in the year 
2000, the target year for measuring the 
effects of reform in meeting the goals of 
the Goals 2000 program, the national and 
state reform initiatives and their 
implications for IPD are discussed. A set 
of reform-oriented approaches for IPD is 
then presented, as are some characteristics 
of effective programs. Part I then 
considers the principles of high-quality 
IPD programs and uses these principles to 
develop a number of items related to 
quality. The final section of Part I 
addresses several data needs for the Year 
2000 National Education Goals Report. 



What Is IPD? 

We first consider several different 
definitions of IPD followed by a brief 
description of the evolution of IPD, 
concluding with the type of IPD needed 
for successful reform. 

Definitions of IPD 

The Department of Education defines 
professional development as including "the 
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rigorous and relevant strategies and 
organizational supports that insure the 
career-long development of teachers and 
other educators." 

It includes preservice preparation and 
training of teachers as well as inservice 
professional development. This paper is 
limited to inservice professional 
development, i.e., to activities designed to 
maintain or upgrade teachers’ professional 
skills following certification or inception 
of teaching including the induction period. 

The Education Information Network in the 
European Union and the EFTA/EEA 
Countries (EURYDICE) defines inservice 
training as "...a variety of activities and 
practices in which teachers become 
involved in order to broaden their 
knowledge, improve their skills and assess 
and develop their professional approach" 
(Perron, 1991). 

A somewhat different definition of IPD 
was proposed by Orlich: "Programs or 
activities that are based on identified 
needs; that are collaboratively planned and 
designed for a specific group of 
individuals; that have a very specific set of 
learning objectives and activities; and that 
are designed to extend, add, or improve 
immediate job-oriented skills, 
competencies, or knowledge" (Orlich, 

1989, p. 5) 

A different view of staff development is 
provided by Odden and Marsh (1988, p. 
598), who are concerned with reform of 
secondary schools: "The emerging mode 
of staff development addresses broader and 
more complex issues, is provided over 
longer time periods with considerable 
ongoing assistance, is linked to strategic 



directions of the district and the school, 
and is targeted to specific issues rather 
than across an array of disconnected 
areas." This mode of staff development is 
not only useful in reform efforts, but, as 
will be seen later in the paper, it is 
consistent with the current consensus of 
IPD experts about the principles of good 
IPD. 

Bellanca (1995) distinguishes among 
inservice, staff development, and 
professional development from the systems 
point of view: 

Inservice is the scheduling of 
awareness programs, usually of 
short duration, to inform teachers 
about a new idea in the field of 
education. 

Staff development is the effort to 
correct teaching deficiencies by 
providing opportunities to learn 
new methods of classroom 
management and instruction. 

Professional development is a 
planned, comprehensive, and 
systemic program designed by the 
system to improve all school 
personnel’s ability to design, 
implement, and assess productive 
change in each individual and in the 
school organization. 

From the individual’s point of view, 
Bellanca notes that "...professional 
development begins with the individual’s 
election to expand his or her repertoire of 
knowledge or skills" in a program "that 
helps the individual understand and do 
higher quality teaching." 
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Evolution of IPD 

The definitions of IPD reflect its 
evolution. As described by Bellanca, 
many years ago inservice opportunities 
were limited primarily to annual institutes 
at which teachers reviewed basic topics for 
annual relicensing. At a later date schools 
and districts introduced the workshops and 
conferences that are now so prevalent. 

Staff development programs differ from 
these inservice events in that these new 
programs required 20 to 30 hours’ study of 
the theory and description of the practice 
(e.g., some of the science and mathematics 
programs that were introduced after 
Sputnik). Many staff development 
programs in the 1980s and the early 1990s 
dealt with cooperative learning approaches 
or with thinking-skills. 

In the early 1990s staff developers began 
to investigate ways to match professional 
development with school improvement; to 
move away from teaching methods that 
might improve learning and to move 
toward management systems that would 
ensure raised test scores. They recognized 
that the constructivists’ insights apply to 
professional development as well as to 
students’ learning. District leaders began 
to understand the power of systemic 
support systems that communicate the idea 
that learning as a lifelong process is as 
important for the teachers as it is for the 
students. 

Today the schools and IPD are being 
shaped by three ideas: results-driven 
education, systems thinking, and 



constructivism (Sparks, 1995). According 
to Sparks these ideas are causing changes 
in IPD. Today IPD is moving towards 
individual development and organizational 
development; it is driven by a strategic 
plan for the school district, each school, 
and the departments that serve schools; it 
is school focused rather than district 
focused; focuses on student needs and 
learning outcomes; involves multiple forms 
of job-embedded learning; focuses on a 
combination of generic and content- 
specific skills; is a major responsibility 
performed by all administrators and 
teacher leaders; is concerned with 
continuous improvement in performance 
for everyone who affects student learning; 
and is an indispensable process for 
preparing students for citizenship and 
productive employment. 

Although some districts are moving in 
these directions, most districts are 
continuing past practice. In the schools 
today we can find all three types of IPD 
defined by Bellanca (inservice, staff 
development, and professional 
development) including programs that are 
mixtures of the types. Therefore SASS 
questionnaires need to cover all of them. 

To avoid confusion, this paper uses the 
term IPD as comprising the three types. 
Although there is no consensus about the 
best type of professional development, the 
view of staff development described by 
Odden and March, Bellanca’ s description 
of professional development, and the 
changed IPD described by Sparks 
correspond to the type of IPD experts 
consider to be essential for successful 
reform. 
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A Framework for Classifying 
Types of IPD Programs 

As the number of approaches to IPD 
proliferates, it becomes increasingly 
important to have a systematic way of 
classifying the approaches in order to 
collect and collate information about IPD 
systematically. A framework for 
classifying IPD types will be useful in 
developing survey items about teachers’ 
staff development activities and in 
analyzing the resulting data. The 
framework should be sufficiently general 
to cover the IPD activities of teachers 
during their induction period as well as 
those of experienced teachers, although 
some of the specific types of activities 
within the framework categories might 
differ for the two groups of teachers. For 
example, during the induction period 
teachers might have a mentor, a program 
of visiting and observing experienced 
teachers, a lighter work load, or regular 
meetings with senior staff and other 
beginning teachers. Experienced teachers 
might take college courses to update their 
knowledge of their subject matter field or 
recent research on pedagogy. 

Before proposing the framework, three 
different approaches will be considered: 
building on the categories used in the 
1993-94 SASS, using the five models of 
staff development proposed by Sparks and 
Loucks-Horsley (1990); or using the six 
research-based models proposed by Gall 
and Vojtek (1994). 

IPD Categories Used in the 1993-94 SASS 

The SASS Teacher Questionnaire included 
the following two questions on types of 
staff development: one concerning 



participation in any of eight activities 
related to teaching and the other 
concerning participation in programs that 
focused on each of five topics and the 
duration (in hours) of the program: 

30. Participation in types of 
inservice activities. Since the end 
of last school year, in which of 
these activities related to teaching 
have you participated? (1) 

SCHOOL DISTRICT sponsored 
workshops or inservice programs, 

(2) SCHOOL sponsored 
workshops or inservice programs, 

(3) University extension or adult 
education courses, (4) College 
courses in your subject field, (5) 
Professional growth activities 
sponsored by professional 
associations, (6) Committee to 
integrate academic skills into 
vocational education, (7) Other 
curriculum committee, (8) 
Committee on selecting textbooks 
or materials, (9) None of the 
above. 

31. Participation in programs with 
a specific focus. Since the end of 
last school year, have you 
participated in any inservice or 
professional development programs 
that focused on the following 
topics? (a) Uses of educational 
technology for instruction (e.g., use 
of computer, satellite learning), (b) 
Methods of teaching your subject 
field, (c) In-depth study in your 
subject field, (d) Student 
assessment (e.g., methods of 
testing, evaluation, performance 
assessment), (e) Cooperative 
learning in the classroom. For 
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each yes answer there is a question 
"How many hours did the program 
last?" with three options: 8 hours 
or less, 9-32 hours, or more than 
32 hours. 

Sparks and Loucks-Horslev Models 

Sparks and Loucks-Horsley (1990) 
proposed five models of staff development. 
The five models were based on their 
analysis of strategies that share similar 
assumptions about "where knowledge 
about teaching practice comes from" and 
"how teachers acquire or extend their 
knowledge". Loucks-Horsley and her 
colleagues (1987) assert that staff 
development programs that are effective in 
changing teachers’ behavior have common 
characteristics. They combine theory and 
application, they provide time for 
reflection and practice and involve self 
study and cooperative learning. The five 
models are described by Loucks-Horsley 
and her colleagues (1989) and Darling- 
Hammond and Cobb (1995): 

Training: e.g., workshops 
sponsored by schools or districts 
where an expert makes a 
presentation focusing on knowledge 
and skills teachers are lacking. 

This is the most common model. It 
includes (1) development of the 
theory and rationale behind the new 
behaviors to be learned, (2) 
demonstration or modeling, (3) 
practice in the training setting, and 
(4) guided practice in the field with 
feedback on performance (Loucks- 
Horsley et al., 1989). 

Individually guided professional 
development : the teacher judges 



what his/her learning needs are and 
establishes a goal. The teacher 
chooses from workshops, library 
research visits, courses offered by 
the district, or may be reimbursed 
for college courses he/she takes, 
and other forms of self study to 
reach the goal. 

Observation/assessment: these forms 
of IPD include clinical supervision, 
peer coaching and teacher 
evaluation with particular attention 
to certain behaviors and open 
discussion of the results. 

School development/improvement 
processes : (This term is used by 
Darling-Hammond and Cobb; 
Loucks-Horsley describes this as 
curriculum and program 
development.) states, districts, or 
schools that try to improve 
education implementing whole- 
school change recognize the 
importance of teachers as agents of 
change. Teachers participate in 
school improvement activities, 
curriculum and assessment 
development, and shared decision- 
making structures. (Little [1993] 
commented that teachers often learn 
more through school development 
processes than through more 
traditional staff development 
activities.) 

Inquiry: includes such activities as 
teacher study groups, teacher 
research, teacher 
collaboratives/networks, or 
reflective inquiry. Such activities 
stem from the reform efforts that 
view the teacher as a guide or 
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facilitator of students’ active 
learning, which forces teachers to 
formulate questions about teaching 
and learning and to inquire both 
into students’ thinking and learning 
and the effects of their teaching. 

Loucks-Horsley and her colleagues (1989) 
provide a detailed description of examples 
of the actual implementation of each of 
these five types of staff development. 

Gall and Voitek Models 

Gall and Vojtek base their six models on 
the objectives of professional development 
described by Sparks and Loucks-Horsley. 
These models are described in terms of the 
different roles for staff developers, and in 
ascending order of their complexity: 

Expert-presenter : teachers assemble 
to listen to an expert talk about a 
topic at professional conferences, 
school district assemblies, 
university courses, and how-to 
workshops. Although this is the 
most prevalent model, it is not 
powerful in itself; it needs to be 
used in conjunction with other 
models. Objectives: development 
of teachers’ knowledge and 
understanding 

Clinical-supervision : the change- 
process supervisor, mentor, or 
coach identifies a teacher’s 
concerns and goals, collects 
classroom observation data, and 
reviews data with the teacher. 
Objectives: development of 
teachers’ instructional skills and 
strategies; development of teachers’ 



ability to reflect and make sound 
decisions 

Skill-training, trainer presents 
theory underlying the skills, 
explains, and models the skills. 
Teacher practices skills and 
receives feedback, is coached to 
promote transfer of training to own 
classroom. (Consistent with the 
constructivist movement in 
education that assumes that 
individuals learn best when they are 
given responsibility for developing 
their own knowledge and 
understanding.) Objectives: 
development of teachers’ (1) 
instructional skills and strategies; 

(2) ability to improve students’ 
academic achievement; (3) ability 
to develop and implement 
curriculum; (4) ability to reflect 
and make sound decisions 

Action-research', teachers do 
research in their own work setting 
to answer their questions or test a 
new idea. Objectives: changing 
teachers’ attitudes; development of 
teachers’ ability to engage in school 
restructuring 

Organization-development', a 
coherent, systematically planned, 
sustained effort at system self-study 
and improvement focusing 
explicitly on change in formal and 
informal procedures, processes, 
norms, or structures, and using 
concepts of behavioral science. 

The goals of organizational 
development are to improve 
organizational functioning and 
performance. Therefore it focuses 
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on groups of teachers and other 
school staff. An organization- 
development specialist helps 
teachers and other staff diagnose 
strengths and weaknesses of their 
school or system, develop a plan of 
action, implement the plan, and 
evaluate its success. Objectives: 
changing teachers’ attitudes; 
development of teachers’ ability to 
develop and implement curriculum 

Change-process: the goal is a 
systemic innovation requiring 
change at the school or district 
level. Staff developers help 
teachers make a decision to adopt a 
systemwide innovation, put the 
innovation into action, and 
institutionalize it. Initiation 
requires staff development to get 
teachers to buy into change; staff 
development required for 
implementation includes "concrete, 
teacher specific training activities, 
ongoing continuous assistance and 
support during the process of 
implementation, and regular 
meetings with peers and others." 
Institutionalization, the decision to 
continue using the systemic 
innovation indefinitely, requires 
staff development to ensure that the 
innovation continues to be used as 
intended— helpful to have teachers 
and other educators who are highly 
skilled in the innovation and who 
can provide training and support to 
new staff. According to Fullan 
(1991), this is by far the most 
complex and lengthy of the models, 
requiring three-to-five years for 
moderately complex changes and 
five-to-ten for major changes. 



Objective: development of 
teachers’ ability to engage in school 
restructuring 

Proposed Framework 

The types of inservice activities in the 
SASS questions are much more specific 
than the categories of types in the two sets 
of models and can easily be fitted into 
either typology. Since it is desirable to 
use general categories in a framework, 
only the Loucks-Horsley and Gall-Vojtek 
models were considered in the proposed 
framework, which consists of seven 
models. The models and the source of 
each model follow: 

Expert-presenter. (Gall and Vojtek) 
This model was and may still be 
the most common form of IPD. It 
has been severely criticized by IPD 
experts as relatively useless for 
reform. Nonetheless it will be 
important to ascertain the extent to 
which it persists in 1998-1999. 

This model was not proposed by 
Loucks-Horsley. 

Skill-training-, (both typologies) 

Observation/assessment-, (both 
typologies) 

Individually guided professional 
development : (Sparks and Loucks- 
Horsley). Gall and Vojtek did not 
include this model since they 
described their models in terms of 
the roles for staff developers. 

Inquiry. (Sparks and Loucks- 
Horsley) This model includes 
action-research, which was a 
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separate model in the Gall and 
Vojtek typology, and encompasses 
many more types of activities, e.g., 
teacher collaboratives/networks, 2 
and reflective inquiry. 

Organization-development'. (Gall and 
Vojtek) This model and the 
following "change-process" model 
are combined by Sparks and 
Loucks-Horsley. They are clearly 
separable, "organization- 
development" corresponding to 
efforts to improve the performance 
of teachers within an existing 
system, and "change-process" to 
changing the performance of 
teachers in a systemic innovation at 
the school or district level. 

Change-process : (Gall and Vojtek) 
Because of the current emphasis on 
systemic reform, it is desirable to 
be able to measure the prevalence 
of "change-process" professional 
development. 

Application of the Framework to 
Develop SASS Items on Prevalence 
of IPD 

Before suggesting specific items, the 
recent work of others related to SASS 
items should be recognized. Mullens 
(1995) reviewed measurement approaches 
for classroom instructional processes. In 
1996, Mullens and his colleagues 
undertook a comprehensive look at the 
theoretical linkages and current 
measurement of student learning, teaching 
quality, and professional development. 
They have released a preliminary draft of 
their work (Mullens et al., 1996) for 



comment. This draft describes the 
research base for linking student learning, 
teaching quality, and professional 
development; discusses professional 
development indicators; proposes a 
typology for the indicators; and reviews 
some 25 surveys for questions that 
correspond to the indicators. When 
agreement on the typology has been 
reached, they plan several additions to the 
draft: a display of the questions from the 
25 surveys; identification of the elements 
of professional development that are 
important, measurable, and representative; 
and a prioritization of these elements. 

Although the framework of models 
proposed in this paper can easily be fitted 
into Mullens’ typology, it has not been 
done because of the typology’s tentative 
state. Instead, suggestions are made for 
SASS to use the proposed framework of 
models to collect data on participation in 
professional development by type and 
related items about the types. Two items 
are suggested to replace items in the 1993- 
94 SASS. 

Prevalence of IPD by Type. Time Teachers 
Spend in Each Type, and Total Dollars 
Teachers Spend on IPD 

Question 30 in the 1993-1994 SASS can be 
expanded to provide time teachers spend 
on each type of IPD. 

Item l: 3 Prevalence of IPD by Type, 

Time Teachers Spend in Each Type, and 
Total Dollars Teachers Spend on IPD. 

The stem might be worded "Since the end 
of the last school year, how many hours 
have you spent in each of the following 
types of staff development? The item 
should list the various types of 
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professional development activities under 
each of the seven major models. By 
providing columns corresponding to time 
intervals in SASS question 31 and 
including a column for zero time, data on 
prevalence of participation in types of IPD 
as well as the time spent in the programs 
can be obtained. At the end of the item, 
add the question "How much of your own 
money have you spent on IPD during this 
school year?" (This last question was 
added as a reaction to Mandel’s (1995) 
statement that "...the extent to which 
teachers meet their employers halfway is 
no less important" than the way schools 
invest their resources for IPD.) 

Question 30 in the 1993-1994 SASS 
provides a list of eight types of 
professional development programs. 

These should be included as subcategories 
of the framework in the question for the 
next SASS to provide trend data. 

The list should include other types that 
have been prevalent in the past (e.g., 
committees dealing with subjects other 
than curriculum, workshops sponsored by 
the school system during the summer, 
skill-training workshops, conference 
attendance, made a presentation at a 
conference or other professional meeting, 
participation in special projects, scheduled 
consultation with colleagues, and 
independent reading). The reform-oriented 
approaches discussed in a subsequent 
section should also be included. Mullens’ 
ongoing review of IPD items in over 25 
educational surveys may also produce 
additional types. The ultimate list will be 
long, but the question should not be too 
burdensome to teachers. 



Another proposal made by Mandel (1995) 
is related to Item 1. He proposed that 
NCES undertake a set of case studies on a 
regular basis that would provide portraits 
of the range of programmatic approaches 
being undertaken in continuing education 
(also in preservice education.) He notes 
the messiness of measuring post-licensing 
education since it takes place in teacher 
centers, colleges and universities, school 
districts, seminars run by disciplinary and 
specialty groups, and in other informal 
settings such as seminars. Nonetheless he 
considers this an arena that is crucial to 
the health of the profession, one that 
deserves much more attention than it has 
received. 

Although such case studies could not easily 
be a part of the SASS surveys, they could 
well be part of the development work that 
would help define SASS questions about 
the range and character of IPD. 

Program Content and Length, and 
Teachers’ Perceptions of Program Impact 

It is possible to build on Question 31 in 
the 1993-1994 SASS to obtain information 
on the content of IPD programs. The 
SASS question obtained information on 
the duration of programs focused on five 
topics. Three of them were topics related 
to current types of methodological 
instruction important in reform: uses of 
educational technology for instruction, 
student assessment, and cooperative 
learning in the classroom. The other two 
were types of knowledge identified by 
Shulman (1986) as necessary for expert 
teaching— content knowledge and 
pedagogical content knowledge. (Shulman 
also named pedagogical knowledge as a 
third type of essential knowledge.). An 
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important topic missing from this set is 
"classroom management skills," described 
by Mullens et al. (1996) as explaining 
rules, monitoring behavior, using 
accountability systems to keep track of 
students’ work, communicating 
expectations clearly, and maximizing the 
amount of class time available for 
academic work. It would also be useful 
for the Year 2000 National Education 
Goals Report to add the topic "teaching 
limited English proficient (LEP) students" 
to the set covered in this question. A 
fuller discussion of this topic is included in 
a later section on the goals report. It 
would also be useful to add topics for 
teaching other types of special student 
populations such as multicultural classes or 
classes that integrate special education 
students. 

It is possible to combine Question 32 with 
Question 31 and obtain teachers’ opinions 
about IPD programs in each of the seven 
topics proposed for Question 31. The 
yes/no participation question in Question 
31 can be eliminated by adding a "0 
hours" category on the right side of the 
question. This leaves space on the left for 
the stub of Question 32 and for three 
columns: agree, no opinion, and disagree. 
Although this sacrifices the more detailed 
scale in Question 32, it has the advantage 
of removing the ambiguity in Question 32 
that was created by not being able to 
differentiate among IPD programs. 

Item 2: 4 Program content and length, and 
teachers’ perceptions of program impact. 
Modify Question 31 by adding "classroom 
management skills" and "teaching limited 
English proficient (LEP) students" to the 
five types of program content, deleting the 
yes/no participation question, and adding 



portions of Question 32 on impact of the 
programs as described above. 

Teachers are not alone in judging the 
quality of IPD programs. A number of 
experts and several organizations have 
provided sets of principles of effective IPD 
programs. 

Education Reform and Teacher 
Inservice Professional Development 

National Reform Initiatives 

Education reform has been pervasive in the 
United States since 1983 when the first 
wave of reform was generated by the 
publication of A Nation at Risk (National 
Commission on Excellence in Education, 
1983). Since preservice and inservice 
professional development are important 
elements of education reform, it is 
important to measure the characteristics 
and prevalence of professional 
development as fully as possible to 
understand the extent of these elements of 
reform. Several reform activities are 
discussed to illustrate the pervasiveness of 
education reform in the United States. 

In 1986, a second wave of reform 
followed the 1983 wave. This second 
wave was stimulated by reports from a 
number of organizations including the 
California Commission on the Teaching 
Profession, the National Governor’s 
Association, the Education Commission of 
the States, the Carnegie Forum on 
Education and the Economy, and the 
Holmes Group. These reports emphasized 
the need to professionalize teaching in 
order to improve education and stem what 
was described as "a rising tide of 
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mediocrity." Renewal of a competent 
teaching force, as well as recruitment, 
preparation, and licensure were now 
recognized as central to educational reform 
efforts (Green, 1987; Darling-Hammond & 
Cobb, 1995). 

These reports stimulated a number of 
initiatives to establish and enforce 
professional standards for teachers: 
professional organizations such as the 
National Science Teachers Association 
established standards for certifying 
members, the National Board for 
Professional Teaching Standards was 
established in 1987 to provide advanced 
professional certification of teachers, the 
20 member states of the Interstate New 
Teachers Assessment and Support 
Consortium (INTASC) developed model 
licensing standards and assessments for 
beginning teachers, and the National 
Council for the Accreditation of Teacher 
Education is reexamining its standards to 
make them consistent with those of 
INTASC and the National Board (Darling- 
Hammond & Cobb, 1995). 

In 1990, President Bush and the nation’s 
Governors established the National 
Education Goals and set a target date of 
the year 2000 for achieving them. This 
constituted a commitment to a nationwide 
effort to reform education around the 
aspirations of the goals (National 
Education Goals Panel, 1995a). With the 
advent of the Goals 2000: Educate 
America Act and the Improving America’s 
Schools Act of 1994, federal funds became 
available for improving teaching. Several 
provisions of the legislation support IPD 
activities. Under the Goals 2000 
legislation, funds for professional 
development are made available to states, 



and states in turn can award subgrants to 
local areas. Under the Improving America 
Schools Act, the Eisenhower Professional 
Development program will support 
sustained long-term IPD efforts related to 
academic standards. In addition, 
provisions in ESEA for disadvantaged 
children and bilingual education include 
funds for professional development. 

Although teacher development was not 
included in the Governors’ six original 
goals, it was added in the Goals 2000 Act 
in 1994, which renumbered the goals 
making the goal for teacher education and 
professional development Goal 4. The 
goal states: 

By the year 2000, the 
Nation ’s teaching force will 
have access to programs for 
the continued improvement 
of their professional skills 
and the opportunity to 
acquire the knowledge and 
skills needed to instruct and 
prepare all American 
students for the next century. 

In 1994, Secretary Richard W. Riley 
established the U.S. Department of 
Education’s Professional Development 
Team to examine research and exemplary 
practices related to professional 
development, to guide the Department’s 
programs and to inform policymakers and 
practitioners across the country. This 
team agreed that "the mission of 
professional development is to prepare and 
support educators to help all students 
achieve to high standards of learning and 
development" (USED, no date). 
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To provide assistance in implementing the 
legislated activities, the U.S. Department 
of Education plans to publish a series of 
idea books to share effective practices with 
educators in carrying out reform efforts. 

The first of the series, Implementing 
Schoolwide Projects: An Idea Book for 
Educators was published in 1994. It 
includes a section on professional 
development that provides a number of 
suggestions for IPD as well as descriptions 
of programs in specific schools. 

Other federal government agencies also 
initiated major programs to reform 
education. For example, in 1991 the 
National Science Foundation initiated a 
Statewide Systemic Initiatives Program 
(SSI) to reform science, mathematics, and 
technology education. During the first 
three years of the program, the Foundation 
signed cooperative agreements with 26 
states to undertake comprehensive reform 
initiatives in these fields, typically over a 
period of five years. The SSI is 
complemented by analogous programs for 
Urban and Rural Systemic Initiatives. 

The SSI programs make heavy demands on 
teachers. "Teachers not only need to 
understand the requirements of the new 
systems, but in many instances, they are 
expected to change their practice, enhance 
their subject-matter knowledge, develop 
new curricula, and serve as overseer and 

assessors in the new process They need 

opportunities to acquire the necessary 
knowledge and skills, to practice new 
strategies, and to interact with other 
teachers about what works and how to 
solve common problems. In short, a 
radically restructured and refocused system 
of professional development is needed. 

The system must be intensive, continuous, 



and connected to classroom practice" 
(CPRE, 1995a, p. 10). Professional 
development is one of the two strategies 
most frequently used by the states for 
changing practice based on the logic that 
changing practice requires changing the 
skills, knowledge, and beliefs of classroom 
teachers (CPRE, 1995b, p. 4). (The other 
strategy is funding local initiatives and 
model schools.) 

As part of an ongoing effort of the NSF to 
increase the impact of its Teacher 
Enhancement Program, the Division of 
Elementary, Secondary, and Informal 
Education started a project known as the 
Local Systemic Change Through Teacher 
Enhancement Project (LSC). This project 
was started because NSF staff recognized 
the need for continuous staff development 
in the schools and the importance of 
working with whole schools instead of 
focusing on individuals if reform is to 
happen. The LSC project consists of a set 
of district-based projects designed to 
reform science, mathematics, and 
technology education through intensive 
upgrading of their K-8 teacher work force. 
In addition to implementing quality 
curriculum materials, the projects must 
provide at least 100 hours of professional 
development in content and pedagogy to 
all participating teachers. This program, 
which began in 1994, has funded 24 
projects (involving 90 districts of varying 
sizes) for up to five years. 

State Reform Initiatives 

States have also initiated (and continue to 
initiate) reforms of teacher education in 
connection with their school restructuring 
efforts. In 1988 The Southern Regional 
Education Board (SREB) surveyed deans 
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of education and deans of arts and sciences 
to determine the changes in the education 
of teachers since 1981. The resulting 
report identified education of teachers as a 
priority in education reform (SREB, 

1988). 

The Education Commission of the States 
(ECS) was also concerned with linking 
teacher education to school reform. State 
leaders expressed dissatisfaction with 
current recertification requirements, noting 
that they were heavy on costly inservice 
activities with little to show for the 
expenditures (Frazier, 1993). In the 
1990s, they expect an increase in 
challenges to the accumulation of random 
course credits that have little significance 
to the teacher or the district. State leaders 
stressed that in outcomes-based systems, 
teacher IPD, whenever possible, should (1) 
be related to making a teacher more 
effective in helping students meet local and 
state goals and (2) should be designed to 
benefit the school and school district in 
reaching organizational goals. ECS 
recognized the need for continuing 
education and recertification of teachers by 
recommending that states "...should 
require recertification programs related to 
individual teacher needs and advancement 
of school and district needs and objectives" 
(Frazier, 1993). The Commission also 
noted the potential of the new professional 
development schools to provide an 
opportunity for higher-quality IPD 
activities than are currently available in 
most districts 

As of July 1995, 49 states and the District 
of Columbia were engaged in standards- 
based education reform (American 
Federation of Teachers, 1995). However, 
experts are critical of the effectiveness of 



current professional development. For 
example Little (1993b) states that "... 
states and districts have been relatively 
slow to reshape professional development 
in ways that respond to the complexities 
and ambiguities of reform. " One of the 
conclusions of CPRE’s 1990 Reform Up 
Close study of high school mathematics 
and science in six states was that there was 
"...little by way of staff development that 
appeared up to the challenges ahead. Most 
staff development we found was 
fragmented and piecemeal, identified and 
delivered by persons distant from the 
classroom, and with little, if any, explicit 
connection to strengthening academic 
instruction" (Porter et al., 1994). Further, 
in discussing the reform of professional 
development Sykes (1996) notes that two 
judgments form the contemporary concern 
for the professional development of 
teachers. The first is that teacher learning 
must be the heart of any effort to improve 
education and the second that conventional 
professional development is sorely 
inadequate. He considers that these two 
judgments represent the most serious 
unsolved problems for American education 
today. He notes the ineffectiveness of the 
"one-shot workshop" in changing what 
goes on in schools and classrooms and 
asserts that the resources for IPD "...are 
too meager and their deployment too 
ineffective to matter. " Although isolated 
efforts are under way to promote teacher 
learning that will lead to improved 
practice, wide-scale efforts have yet to 
emerge. With the many reform initiatives 
under way and the extensive professional 
development that will take place between 
the 1993-94 SASS and the 1998-1999 
SASS, it is extremely important that NCES 
measure change in this activity and its 
extent and effects as fully as possible. 
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Implications of Reform for IPD 

Little (1993a) has noted that most current 
reform initiatives fit into one or more of 
five streams of reform, all of which 
present challenges to teachers: 

• Reforms in subject-matter teaching 
(standards, curriculum, and 
pedagogy), 

• Reforms centered on problems of 
equity and the increasing diversity 
of the student population, 

• Reforms in the nature, extent, and 
uses of student assessment, 

• Reforms in the social organization 
of schooling, and 

• Reforms in the professionalization 
of teaching. 

These reforms call for major 
improvements in students’ outcomes 
including critical thinking (which may not 
be part of the teachers’ current practice); 
identifying and altering classroom practices 
that contribute to student failure; authentic 
assessment, although teachers may not 
have the skills to design and implement 
such assessment; and school restructuring 
that may be based on principles rather than 
practices, without models to translate the 
principles into instructional strategies. 

Most of the existing resources for 
professional development that are limited 
to skills training are not ready to meet the 
demands of these reforms that call for 
expanding teachers’ opportunities to learn, 
experiment, consult, and evaluate. This 
does not imply that there is no longer a 
role in professional development for the 



thoroughly tested models of skill training 
with opportunities for classroom practice 
and classroom coaching and/or 
consultation. Skill development models 
can be very effective for training related to 
specific transferable skills and new ideas. 

Today most IPD is carried out by school 
districts. It consists of formal education 
activities such as workshops, inservice 
programs lasting a day or a half day at 
which experts lecture and that may include 
each teacher’s choice of workshops led by 
trainers. The programs may provide 
material or suggestions that are useful to 
the teachers, but there is seldom follow-up 
to evaluate the effectiveness and utility of 
the programs. Typically they have little 
effect on practice because they lack focus, 
intensity, follow-up, continuity, and 
linkage with the district’s goals for student 
performance. (Corcoran, 1995). Another 
common form of IPD is highly theoretical 
university coursework; half of all teachers 
reported earning college credits during the 
period 1988-91 (NEA, 1992). But these 
are not the types of IPD that will meet the 
demands of reform. In discussing the 
condition of teaching in America today, 
Darling-Hammond (1995) notes that 
although attempts are presently under way 
across the country to make a strategic 
investment in the professional development 
of teachers, they are embryonic and 
scattered rather than systematic. She 
recognizes, however, that "... the 
possibilities for rethinking how schools 
structure the use of teacher time, the 
opportunities for team teaching and 
collaboration, the development of teacher 
and school networks, and the 
responsibilities of teachers are probably 
greater now than they have ever been. " 
These opportunities constitute some of the 




107 



characteristics of good professional 
development. The next section considers 
this topic more extensively. 

Reform-Oriented Approaches for 
Professional Development 

Corcoran (1995, pp. 5-6) describes seven 
relatively new approaches to professional 
development that may be effective in 
reform and comments (paraphrased) on 
their desirable characteristics: 

Joint work— shared responsibility 
for tasks such as team teaching, 
curriculum committees, or other 
jobs that create independence 
among teachers and require 
cooperation. (Provides 
opportunities for exchange among 
teachers and reflection about 
practice.) 

Job enrichment— expansion of 
teachers’ work in ways that require 
new skills, such as the scoring of 
portfolios in Vermont or serving as 
mentors to beginning teachers. 
(Provides opportunities for teachers 
to discuss their practice and share 
ideas.) 

Teacher networks— focus on specific 
subject-matter and seek to deepen 
teachers’ understanding of content 
and their facility with new teaching 
strategies. (Offer access to a 
"professional community" and 
discourse about improving 
practice.) 

Collaborations between schools and 
colleges— often required to fill need 
for professional development of 



sufficient intensity. (Helps teachers 
meet the requirements of reforms 
for deeper knowledge of subject 
matter.) 

Professional development (or 
practice schools)-although 
primarily used in preservice 
development, they could bring 
novice and experienced teachers 
together with university clinical 
faculty to improve their practice 
through observation, low-risk 
experimentation, reflection, and 
coaching. 

National board certification— the 
process of applying for certification 
is thought to be excellent 
professional development since it 
requires teachers to document their 
practice, reflect on their strengths 
and weaknesses, and demonstrate 
specific knowledge and skill. 

Teachers as researchers— research in 
classrooms and schools in 
cooperation with their colleagues 
and university faculty. Frequently 
directed at problems identified by 
teachers, or may be defined by 
academic interests. (Benefits: 
stimulate discussions, help 
organizations define problems, and 
lead to changes in practice and 
policy.) 

These approaches are consistent with the 
principles of high-quality professional 
programs that are discussed in a later 
section. They also share some common 
characteristics: 



• They respect the expertise of 
accomplished teachers 

• They are integrated with teachers’ 
work 

• They are based on current research 
on teaching and learning 

• They recognize teachers as a 
valuable source of information 
regarding effective professional 
development and include them in its 
design and implementation 

Little (1993a, pp. 4-5) also addresses 
alternatives to traditional approaches. 

These alternatives are ones "...that engage 
teachers in the pursuit of learning in ways 
that leave a mark on their perspectives and 
their practice." She describes four 
alternative models, the first two being ones 
that were also listed by Corcoran: 

Teacher collaboratives and other 
networks— subject-specific teacher 
collaboratives share the view that 
teachers’ professional development 
encompasses (1) teachers’ 
knowledge of academic content, 
instruction, and student learning; 

(2) teachers’ access to a broader 
network of professional 
relationships; and (3) teacher 
leadership in the reform of 
systemwide structures. 
Collaboratives underscore teachers’ 
involvement in the construction of 
subject matter knowledge. Thus 
they prepare teachers to make 
informed responses to reforms in 
subject matter teaching and student 
assessment. 



School-University collaborations 
targeted at school reform— on the 
whole, these partnerships have 
formed between individual activists 
in universities and schools or 
districts, or between individual 
consultants and schools, or between 
departments of education and local 
schools. They have not routinely 
incorporated faculty from subject 
matter departments. They hold 
promise as vehicles for more 
effective professional development, 
e.g., insider/outsider attached to the 
school to provide support, expand 
access to resources and to critique 
school progress, e.g., The Coalition 
of Essential Schools. Other 
partnerships such as the Chicago 
Project on Learning and Teaching 
have the goal of promoting 
breakthroughs in conceptual 
understanding for the teachers and 
to immerse them in math 
experiences. 

Subject matter associations— clearly 
they are exerting increasingly 
powerful influences in the design of 
subject curriculum and assessment 
standards. They are positioned to 
exert strong influence on teachers’ 
dispositions toward reform 
proposals. Their effect may be 
multiplied if the association’s most 
active members also occupy 
leadership roles within their school, 
district, or union. 

Special institutes and centers— 
teachers say they provide a good 
professional development 
experience. They offer great depth 
and focus, enough time to grapple 
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with ideas and materials, the sense 
of doing real work rather than 
being talked at, and an opportunity 
to consult with colleagues and 
experts. (They also cost more per 
participant, and are less accessible 
than more modest local programs.) 
Teachers enjoy the opportunity for 
sustained work with ideas, 
materials, and colleagues. 

The approaches described above can be 
effective only if the structures exist to 
make them available to teachers and to 
provide them support for classroom 
implementation of what they have learned. 
The three common characteristics of good 
staff development structures are identified 
by Loucks-Horsley et al. (1989, pp. 45- 
48): 

Support for the practice and 
refinement of new behaviors in the 
classroom 

Opportunities for teachers to talk 
and work together to reinforce, 
problem solve, and encourage 
change 

A clear message that the new 
behaviors are important and 
teachers are expected to use them 

They describe several types of effective 
staff development structures for elementary 
science: institutes similar to NSF- 
sponsored institutes of the past, teachers 
centers, and networks and partnerships, all 
of which were mentioned by either 
Corcoran or Little. 



Principles of High-Quality 
Professional Development 
Programs 

How often do you hear 
statements to the effect that 
the continuous professional 
development of teachers is 
the key to school 
improvement? ...the general 
endorsement of inservice 
education means nothing 
without an accompanying 
understanding of the 
characteristics of effective as 
compared with ineffective 
inservice education efforts. 

Nothing . . . has promised so 
much and has been so 
frustratingly wasteful as the 
thousands of workshops and 
conferences that led to no 
significant change in 
practice when the teachers 
returned to their classrooms 
(Fullan, 1991). 

Although successful school reform requires 
many ingredients, the one essential 
ingredient is the classroom teacher. The 
Goal 4 Resource Group of the National 
Education Goals Panel is well aware that 
only recently have we fully appreciated the 
ways of teaching complex subject matter to 
diverse students and consequently many of 
our current teachers are under- or 
unprepared. They provide a compact 
definition of high-quality IPD: 

"Professional development should be 
continuous, sustainable, site-based, context 
driven, focused on student learning and 
designed to promote school-wide 
innovation and change" (NEGP, 1995c). 
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Corcoran (1995) states that the 
reform movement will require a shift from 
a behaviorist approach to teaching 
"...approaches which actively engage 
students in the construction of knowledge." 
A number of experts and organizations 
have discussed principles and policies for 
professional development programs that 
are consistent with the current reform 
efforts. Corcoran (1995) summarizes their 
work and provides a list that is based on 
the work of G. Griffin (1982), B. Joyce 
and B. Showers (1982), S. Loucks- 
Horsley, C. Harding, M. Arbuckle, L. 
Murray, C. Dubea, and M. Williams 
(1987), N. L. Zimpher and K. R. Howey 
(1992), J. W. Little (1993), H. Price 
(1993), National Staff Development 
Council (1994), and H. Hodges (1994). 
Their suggestions include programs that 
incorporate the following principles or 
policies: 

Stimulate and support site-based 
initiatives. Professional 
development is likely to have 
greater impact on practice if it is 
closely linked to school initiatives 
to improve practice. 

Support teacher initiatives as well as 
school or district initiatives. These 
initiatives could promote the 
professionalization of teaching and 
may be cost-effective ways to 
engage more teachers in serious 
professional development activities. 

Are grounded in knowledge about 
teaching. Good professional 
development should encompass 
expectations educators hold for 
students, child-development theory, 
curriculum content and design, 



instructional and assessment 
strategies for instilling higher-order 
competencies, school culture and 
shared decision making. 

Model constructivist teaching. 
Teachers need opportunities to 
explore, question, and debate in 
order to integrate new ideas into 
their repertoires and their 
classroom practice. 

Offer intellectual, social, and 
emotional engagement with ideas, 
materials and colleagues. If teachers 
are to teach for deep understanding, 
they must be intellectually engaged 
in their disciplines and work 
regularly with others in their field. 

Demonstrate respect for teachers as 
professionals and as adult learners. 
Professional development should 
dtaw on the expertise of teachers 
and take differing degrees of 
teacher experience into account. 

Provide for sufficient time and follow- 
up support for teachers to master 
new content and strategies and to 
integrate them into their practice. 

Are accessible and inclusive. 
Professional development should be 
viewed as an integral part of 
teachers’ work rather than as a 
privilege granted to "favorites" by 
administrators. 

Little (1993a) also discusses principles and 
adds three: 
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• Should take explicit account of the 
contexts of teaching and the 
experience of teachers 

• Should offer support for informed 
dissent 

• Should place classroom practice in 
the larger contexts of school 
practice and the educational careers 
of children 

The U.S. Department of Education’s 
Professional Development Team also 
developed a set of principles (1995). 

Their principles reflect related research 
and exemplary practices and the review 
and comments on the principles by a large 
number of people and organizations. The 
team provided ten principles: 

• Focuses on teachers as central to 
student learning, yet includes all 
other members of the school 
community 

• Focuses on individual, collegial, 

and organizational improvement 

• Respects and nurtures the 
intellectual and leadership capacity 
of teachers, principals, and others 
in the school community 

• Reflects best available research and 

practice in teaching, learning, and 
leadership 

• Enables teacher to develop further 

expertise in subject content, 
teaching strategies, uses of 
technologies, and other essential 
elements in teaching to high 
standards 



• Promotes continuous inquiry and 
improvement embedded in the daily 
life of schools 

• Is planned collaboratively by those 
who will participate in and facilitate 
that development 

• Requires substantial time and other 
resources 

• Is driven by a coherent long-term 
plan 

• Is evaluated ultimately on the basis 
of its impact on teacher 
effectiveness and student learning; 
and this assessment guides 
subsequent professional 
development efforts 

Although the Department’s list repeats 
some of those suggested by Corcoran and 
Little, there are no inconsistencies among 
the principles. 

More recently, Howley and Valli (1996) 
proposed another set of principles for 
effective professional development, which 
they named "the consensus model of 
professional development. The model is 
based on the implications of recent 
research on learning for professional 
development. They first summarize the 
convergence of research on learning 
reported in several recent syntheses of 
such research. Five "learner-centered 
principles" of learning have been 
identified: 

• One’s existing knowledge serves as 
a foundation of all future learning 
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• The ability to reflect upon and 
regulate one’s thoughts and 
behaviors is essential to learning 
and development 

• Motivational or affective factors 
along with the motivational 
characteristics of the learning tasks 
play a significant role in the 
learning process 

• Learning processes through various 
common stages of development 
influenced by both inherited and 
experiential/environmental factors 

• Learning is as much a socially 
shared undertaking, as it is an 
individually constructed enterprise 

This research on learning has stimulated a 
number of new studies of professional 
development that reach remarkably 
consistent conclusions "...with respect to 
the characteristics of professional 
development that are most likely to lead to 
improvements in actions of educators that 
contribute to student learning." Based on 
these studies, Hawley and Valli propose 
their new consensus model of professional 
development with eight design principles: 

• Driven, fundamentally, by analyses 
of the differences between (a) goals 
and standards for student learning 
and (b) student performance 

• Involves learners (e.g., teachers) in 
the identification of their learning 
needs and, when possible, in the 
development of the learning 
opportunity and/or the process to 
be used 
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• Is primarily school-based and 
integral to school operations 

• Provides learning opportunities that 
relate to individual needs but are, 
for the most part, organized around 
collaborative problem solving 

• Is continuous and ongoing, 
involving follow-up and support for 
further learning— including support 
from sources external to the school 

• Incorporates evaluation of multiple 
sources of information on (a) 
outcomes for students and (b) 
processes that are involved in 
implementing the lessons learned 
through professional development 

• Provides opportunities to engage in 
developing a theoretical 
understanding of the knowledge and 
skills to be learned 

• Is integrated with a comprehensive 
change process that deals with the 
full range of impediments to and 
facilitators of student learning 

Hawley and Valli’s list is further evidence 
of the consensus among researchers— all 
but one (the fourth) of their principles are 
included in the lists considered earlier. 
Hawley and Valli not only provide 
illustrative references that support each of 
the principles in their model, but they 
document the research base for this list by 
tabulating the relationships between the 
five learning principles and the design 
principles of the consensus 
model of professional development. 



113 



120 



Use of the Principles to Develop 
SASS Items Related to IPD 
Quality 

The list of principles is heterogeneous— it 
contains aspects of the planning and 
coordination of IPD; ways in which 
schools organize to facilitate and stimulate 
teacher learning; the growth opportunities 
being provided for teachers; school 
support for professional development; and 
school environment. Each of these topics 
should be explored in SASS to study the 
extent to which current IPD programs are 
consistent with the principles of high- 
quality professional development 
programs. 

Information about school-based IPD 
programs could be obtained by adding 
questions to the Principal Questionnaire 
and Teacher Questionnaire. Teachers 
could also provide information about the 
off-site IPD in which they participate. 

Planning and Coordination of Professional 
Development 

Two items are suggested related to 
planning and coordination of professional 
development. The first pertains to 
elements of effective planning and 
coordination of IPD. The second to 
reasons why teachers choose not to 
participate in IPD. Although the second 
item addresses neither planning nor 
coordination, it provides information 
needed by policymakers to take 
appropriate corrective actions, actions that 
may include improvement of their planning 
and coordination of IPD. 

Elements of planning and coordination of 
IPD. An item on effective elements of 



planning and coordination of IPD should 
be added to the Principal Questionnaire. 
The following suggestions for the item are 
drawn from Corcoran’s framework for 
reviewing IPD policies and practices 
(Corcoran, 1995), from expressed needs of 
the Goal 4 Resource Group of the National 
Education Goals Panel (1995), and from 
the principles published by the U.S. 
Department of Education’s Professional 
Development Team. (Questions have been 
added that are not in the above list of 
principles and some that are included in 
the list have been reworded.) 

Item 3: 5 Planning and Coordination of 
IPD. Provide columns for answering yes 
or no to each question. 

• Is there a state plan for IPD and 
are there state priorities? 

• Does the state or district require 
that schools develop plans? 

• Are IPD activities tied to school 
improvement? 

• Is there coordination among 
providers of IPD? 

• Are teachers required to develop 
professional improvement plans? 

• Are teachers involved in the 
development of the learning 
opportunity and/or the process to 
be used? 

• Are teacher salary increments 
dependent on the job relatedness of 
IPD activities? 

• Are state initiatives to set standards 
and develop curriculum frameworks 
and new assessments supported by 
appropriate professional 
development? 

• Is your school or school district 
engaged in partnerships that will 
promote community stakeholders’ 
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support of programs for 
professional development of 
educators? 

Reasons for nonparticipation I PD programs. 
Little (1993b) discusses the wide variation 
in profiles of participation in IPD by 
teachers with comparable experience and 
teaching assignments. She notes that these 
differences persist even in schools formally 
committed to reform initiatives. She 
illustrates this point with data from the 
Illinois Writing Project in which less than 
half the teachers in urban schools attended 
the after-school workshops. Understanding 
why teachers choose not to participate in 
IPD programs is important to policymakers 
so they can take appropriate action to 
increase participation. Such information 
could be obtained by adding an item to the 
Teacher Questionnaire for teachers who 
have not participated in IPD during the 
prior year. The options in this item are 
paraphrased from Little’s specific 
illustration. 

Item 4: 6 Reasons for Nonparticipation in 
IPD Programs. Select up to three reasons 
why you chose not to participate in IPD 
programs during this school year. Enter 
" 1 " for the most important reason, if you 
select two or three reasons enter "2" for 
the next most important, if you select three 
reasons, enter "3" for the least important 
reason. 

Priority 

Unimpressed with the quality of 

the program 

Already expert in the practices 

of the program 

Pressed by the demands of too 

many projects 



Teaching load too burdensome 

Committed to other activities that 

required my time, thought, and 
energy 

Not persuaded that participation 

would make a difference to my 
students 

Discouraged by failures of 

administrative leadership 
Truly discouraged about teaching 

School Organization for Teacher Learning 
and Other Growth Opportunities Provided 
for Teachers 

An item on the ways schools organize for 
teacher learning by integrating teacher 
development into the daily activities of 
teaching and other growth opportunities 
provided for teachers should be added to 
the Principal Questionnaire. The questions 
are derived from Corcoran’s framework 
for reviewing professional development 
policies and practices and the Goal 4 
Resource Group. They also stem from 
two of the six aspects of school 
organization identified by Little (1996) as 
related to teachers’ learning and 
professional development: (1) extent of 
collective focus on students and shared 
responsibility for student learning, and (2) 
teacher assignment policies and practices 
that satisfy criteria of fit, stretch, and 
community. (The questions include some 
not in the above list of principles and some 
that are reworded.) 

Item 5: 7 School Organization for Teacher 
Learning and Other Growth Opportunities 
Provided for Teachers. (Provide columns 
for answering yes or no to each question.) 

• Are growth opportunities built into 
teachers’ workdays? 
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Do teachers have regular 
opportunities to work together? 

Do teachers have a high level of 
collective responsibility for student 
learning? 

Do teachers engage in systematic, 
sustained, collective study of 
student work— coupled with a 
collective effort to figure out the 
roots of student work in the 
practice and choices of teaching? 
Does school policy support the 
individual and collaborative 
investigation of selected problems 
and questions that arise in teaching? 
Is it school policy to develop the 
organizational habit of shared 
student assessment? 

Are teacher assignments based on 
making the best use of an 
individual teacher’s existing 
knowledge, experience, and 
interest? 

Are teacher assignments based on 
stretching teachers’ understanding 
and skill as well as using their 
existing expertise? 

Are teaching assignments designed 
to configure a staff in ways that 
provide a basis for professional 
exchange, mutual support, or 
shared inquiry? 

Are teachers performing 
professional or administrative tasks 
requiring significant skills? 

Is support provided for beginning 
teachers? 

Does your school district support 
teachers who are seeking National 
Board Certification? 

How much time is set aside for 
professional development during the 
school year? (Provide three time 
options.) 



• Do all teachers have full and equal 
access to high-quality IPD 
activities? 

• Do these opportunities vary across 
grade levels? 

• Do the state colleges and 
universities provide appropriate 
courses accessible to all teachers? 

• Does your school have sustained 
partnerships with other 
organizations (e.g., professional 
development schools or professional 
associations) that provide a home 
for professional development 
options? 

Support for IPD 

In addition to embedding teacher learning 
in the daily work of teaching and 
providing other opportunities for IPD, 
schools and districts can provide several 
types of support for IPD. They can 
provide incentives, time, and support for 
teachers to participate in IPD programs. 

Incentives to participate in IPD. 

Policymakers need to balance individual 
and organizational interests in IPD, and to 
provide incentives so they are aligned. 

The arrangements for IPD should support 
schoolwide improvement and at the same 
time stimulate the teacher’s professional 
growth and engagement in teaching, and 
support career advancement (Corcoran, 
(1995, p. 6). An item should be added to 
the Principal Questionnaire on incentives. 
Again, the options in the question come 
from Corcoran and Little. 

Item 6: 8 Incentives to Participate in IPD. 
What incentives are provided for teachers 
to participate in professional development 
and to improve their practice? (Provide 
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columns to answer yes or no to each 
question.) 

° Is professional development linked 

to personnel evaluation and 
recertification? 

° Do districts reimburse college 

tuition for graduate study? 

° Are salary increments linked to 

professional development? 

0 Does professional growth bring 

increased responsibility, status, or 
recognition? 

° Are school resources available for 

teachers to participate in 
professional community and 
personal endeavors beyond the 
school? 

It would also be useful to ask the question, 

° How do the incentives affect 

teachers in different grade levels, 
or career stages? 

but this would require a separate item with 
a different structure. 

Providing time for professional development. 
In the Foreword to the publication 
Breaking the Tyranny of Time: Voices 
from the Goals 2000 Teacher Forum 
(USED, 1994) the Secretary of Education, 
Richard W. Riley, describes the critical 
element of time as one of the greatest 
issues in education reform. The teachers 
who participated in the forum, 119 in 
number, identified time as the most critical 
resource for the success of school reform. 
It is no surprise that one of the eight 
recommendations of the conference was 
"We recommend that teachers be provided 
with the professional time and 
opportunities they need to do their jobs. " 



Corcoran (1995) discusses one of the steps 
policymakers should be taking to improve 
professional development for teachers— 
increasing the time available for teacher 
interaction and professional development. 
He summarizes five approaches described 
by Watts and Castle (1993) that have been 
used to increase the time available for 
IPD: 

Using substitutes or releasing 
students. Some schools are 
effectively using one morning or 
afternoon a week for teacher 
development and other 
improvement activities. However, 
this approach provides only small 
blocks of time and is often resented 
by parents. 

Purchasing teacher time by using 
permanent substitutes, retirees, or 
giving compensation for weekends 
or summer work. This is 
expensive, sporadic, and some 
teachers will not participate on 
weekends or during the summer. 

Scheduling time by providing 
common planning time for teachers 
working with the same children or 
teaching the same grade on a 
regular basis. This is often done in 
schools using instructional teams, 
but it could be done in many more 
schools if assistance was provided 
with block scheduling. 

Restructuring time by permanently 
altering teaching responsibilities, 
the teaching schedule, school day, 
or school calendar. This has 
serious implications for busing, 
union contracts, facilities 
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maintenance, state regulations, and 
budgets. It also means changing 
public expectations— a reason few 
schools or districts have taken this 
approach. 

Making better use of available time 
and staff. Decrease the hours 
teachers spend in the classroom to 
provide them with more time for 
professional work. Although 
costly, the costs could be 
minimized by: 

Occasionally substituting 
appropriate television 
programming for regular 
instruction; 

Using adult volunteers or 
older students to provide 
extracurricular activities for 
children; 

Using occasional large 
classes for special topics, 
for exposure to the arts, or 
presentations of outside > 

"experts"; 

Using independent study to 
let students pursue projects 
on their own: and/or 

Involving more students in 
community service 
activities. 

An item should be added to the Principal 
Questionnaire asking what actions have 
been taken to provide teachers with more 
time for professional development based 
on the approaches described above. 



Item 7: 9 Providing Time for Professional 
Development. Which of the following 
actions (a list developed from the five 
approaches described above) have been 
taken in your school to increase time 
available to teachers for IPD. (Columns 
should be provided for answering yes or 
no to each action.) 

Support for IPD in main teaching assignment 
field. In Question 33 of the 1993-94 
SASS, the first two questions pertain to 
providing time for IPD and the other 
questions refer to monetary support or 
support for professional growth credits. 
Although this question appears to overlap 
the proposed Item 7, this question is 
limited to IPD related to the teacher’s 
main teaching assignment field and is 
addressed to teachers rather than to the 
principal, which makes it possible to look 
at equity among groups of teachers in the 
allocation of these types of support. 
Therefore this question should be repeated, 
but with the addition of an item for "leaves 
or sabbaticals." 

Item 8: 10 Support for IPD in Main 
Teaching Assignment Field. Repeat 
Question 33 of the 1993-94 SASS, but 
starting with the addition of an item for 
"sabbaticals and leaves" and a change in 
the wording of the current first item to 
read "other released time from teaching." 

School Environment 

Although high-quality professional 
development programs that influence the 
knowledge and abilities of teachers are 
important, teachers also need to work in 
an environment that is supportive of good 
teaching. Research on educational quality, 
teacher professionalism, policy 



implementation, effective schools, and 
educational change suggests that several 
characteristics of the school environment 
are related to effective education (Loucks- 
Horsley et al., 1989): 

® Clear purposes and outcomes 

® Adequate, appropriate resources, 
including time, staff, and materials 
® A robust conception of staff 

development 

• Norms of experimentation, risk 
taking, collegiality and 
collaboration 

• Involvement in decision making 

• Leadership and support 

The importance of these characteristics of 
school environment were recognized in the 
development of the 1993-94 SASS in three 
of the questions about teachers’ 
perceptions and attitudes toward teaching: 

Question 44. At this school, how 
much actual influence do you think 
teachers have over school policy in 
each of the following areas? (See 
Appendix A for the list of areas 
and the scale used in this question 
and the following question.) 

Question 45. At this school, how 
much control do you feel you have 
IN YOUR CLASSROOM over 
each of the following areas of your 
planning and teaching? 

Question 47. Do you agree or 
disagree with each of the following 
statements? (A list of 25 statements 
that relate to the six characteristics 
of school environment listed by 
Loucks-Horsley follows.) 



These three questions should be repeated 
in the 1998-1999 SASS to make it possible 
to measure the major changes in school 
environment that can be expected between 
1993-94 and 1998-1999 due to the 
systemic reform efforts under way in many 
schools and districts. These efforts can be 
expected to lead to (1) increased 
professionalization of teachers and (2) 
schools, classrooms, and teachers that 
value questions, experimentation, risk 
taking and collaborative problem solving. 
Both of these results are dependent in part 
on school environment. 

Some augmentation of the questions, 
particularly Question 47, might be 
desirable. The report by Mullens et al. 
(1996) included a review of the items on 
professional development in over 25 
surveys. They report finding six surveys 
that include items on 32 elements thought 
to affect school culture. It would be 
desirable to match these 32 elements 
against those used in the SASS questions 
to look for possible additions to the SASS 
questions. It would also be desirable to 
group the statements for each of Loucks- 
Horley’s six characteristics so that it 
would be easier for teachers to understand 
the purpose of the question. 

Item 9: 11 Teachers’ Influence Over 
School Policy. Repetition of SASS 
Question 44. 

Item 10: 12 Teachers’ Control in the 
Classroom of Planning and Teaching. 
Repetition of SASS Question 45. 

Item ll: 13 Teachers’ Perceptions of 
School Environment. Repetition of SASS 
Question 47 with items grouped by the 
Loucks-Horsley characteristics and with 
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possible additional items identified in 
research. 

Data Needs for the Year 2000 
National Education Goals Report 

In 1994, the Goals 2000 legislation 
formally authorized the National Education 
Goals Panel (NEGP), a bipartisan 
committee of state and federal officials that 
had been meeting since 1990 to monitor 
progress toward the goals. Charges to the 
panel included continuing to play a major 
role in tracking education reform through 
its annual reports on progress toward 
meeting the education goals. The 1995 
goals report (National Education Goals 
Panel, 1995b), which is the Panel’s fifth 
report, includes a section on teacher 
education and professional development 
that lists the four objectives under goal 4: 

All teachers will have access to 
preservice teacher education and 
continuing professional 
development activities that will 
provide such teachers with the 
knowledge and skills needed to 
teach to an increasingly diverse 
student population with a variety of 
educational, social, and health 
needs. 

All teachers will have continuing 
opportunities to acquire additional 
knowledge and skills needed to 
teach challenging subject matter 
and to use emerging new methods, 
forms of assessment and 
technologies. 

States and school districts will 
create integrated strategies to 



attract, recruit, prepare, retrain, 
and support the continued 
professional development of 
teachers, administrators, and other 
educators, so that there is a highly 
talented work force of professional 
educators to teach challenging 
subject matter. 

Partnerships will be established, 
whenever possible, among local 
educational agencies, institutions of 
higher education, parents, and local 
labor, business, and professional 
associations to provide and support 
programs for the professional 
development of educators. 

The report also provides nine measures of 
progress toward the goal of which three 
deal with professional development and 
two with teacher support. The other 
measures relate to preparation to teach 
limited English proficient students and 
preservice teacher education and 
certification. All of the measures were 
derived from the Teacher Questionnaires 
in the 1991 and 1994 School and Staffing 
Surveys. In fact the NCES expanded the 
section on staff development in the 1993- 

1994 SASS to provide information needed 
by NEGP. 

Enhancements for the Year 2000 Goals 
Report Derived from the Suggested Items 

The items that have been suggested for 
inclusion in the 1998-1999 SASS would 
have a very positive effect on the goals 
report for the year 2000 by providing 
additional detail for measures used in the 

1995 report as well as a number of 
additional measures. Each of the five 
measures on professional development and 
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teacher support in the 1995 report are 
discussed in turn showing the source of the 
information used in the 1995 report and 
the effect the suggested items could have 
for the year 2000 report. 

The three measures of professional 
development are direct measures of the 
goal: 

Participation in Professional 
Development Activities on 
Selected Topics: Percentage of 
teachers who reported that they 
participated in inservice or 
professional development programs 
on various topics (uses of 
educational technology, methods of 
teaching subject field, in-depth 
study in subject field, and student 
assessment) since the end of the 
previous school year, 1994. 
Participation is tabulated for all 
teachers, and for urban, suburban, 
and rural teachers. 

(Source: SASS Question 31) 

Item 2 would add two topics to the list in 
Question 31. Both of these topics are 
important in reform: classroom 
management skills and preparation to teach 
limited English proficient (LEP) students. 

It would also provide information on the 
teachers’ opinions about the impact of the 
programs. 

Support for Professional 
Development: Percentage of 
teachers who reported that they 
received various types of support 
(released time for teaching or 
scheduled time, travel, per diem 
expenses, tuition, and/or fees: and 
professional growth credits). 



Support is tabulated for all 
teachers, and for urban, suburban, 
and rural teachers. 

(Source: SASS Question 33) 

Item 8 would add one additional type of 
support "sabbaticals and leaves" to the list 
in the 1993-94 SASS. 

Participation in Different Types 
of Professional Development 
Activities: Percentage of teachers 
who reported that they participated 
in various activities related to 
teaching (workshops or inservice 
programs, college courses, and 
activities sponsored by professional 
associations) since the end of the 
previous school year, 1994. 
Participation is tabulated for all 
teachers, and for beginning 
teachers, teachers with four-to-ten 
years of experience, and teachers 
with more than ten years of 
experience. 

(Source: SASS Question 30) 

Item 1 would greatly expand the list of 
types of IPD activities. Of special 
importance, it would include new 
approaches for IPD that may be effective 
in reform. In addition it would provide 
information on the amount of time spent in 
each program and the teachers’ total 
monetary expenditure for IPD. 

The two measures that deal with teacher 
support are direct measures of the third 
objective and are closely related to 
inservice professional development: 

Support through Formal Teacher 
Induction Programs: Percentage 
of teachers (by experience 
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categories) who reported that 
during their first year of teaching, 
they had participated in a formal 
teacher induction program to help 
beginning teachers by assigning 
them to master or mentor teachers, 
1994. Information is provided for 
the following categories of 
teachers: all, elementary, 
secondary, urban, suburban, and 
rural. Information is also provided 
on change in participation in 
induction programs between 1991 
and 1994. 

(Source: SASS 1993-94 Question 
35a and SASS 1990-91 Question 
28a) 



As indicated earlier, items about induction 
programs for the 1998-1999 SASS will be 
the subject of a subsequent paper. 

Teacher Influence over School 
Policy: Percentage of teachers who 
reported that teachers in their 
school have influence over school 
policy in selected areas 
(determining the content of 
inservice programs, establishing 
curriculum, and setting discipline 
policy). Information is provided 
for the following categories of 
teachers: all, elementary, 
secondary, urban, suburban, and 
rural. Information is also provided 
on the change between 1991 and 
1994 in teacher influence over 
school policy for the selected areas. 
(Source: SASS 1993-1994 
Question 44 and SASS 1990-91 
Question 39) 



Since Item 9 is a repetition of Question 
44, it will provide no additional 
information. 

In addition to the above items on 
professional development and teacher 
support, the NAGB report includes two 
measures that deal with preparation to 
teach limited English proficient (LEP) 
students: (1) percentage of teachers who 
reported that they have LEP students in 
their classes and have received training to 
teach LEP students and (2) percentage of 
teachers who reported that they have 
received training to teach LEP students, 
1994. The SASS question on training does 
not inquire when the training was received 
so it is not clear whether the training was 
preservice or inservice. The suggestion in 
Item 2, to add a topic on preparation to 
teach LEP students to the list of topics of 
IPD programs in Question 31 of the 
Teacher Questionnaire, would eliminate 
this ambiguity and help measure current 
efforts for the first objective of Goal 4. 

As mentioned, it would also be possible 
for Item 2 to include topics related to 
teaching other special student populations, 
which would provide even more 
information relevant to the first objective. 

Additional Measures for the Year 2000 Goals 
Report 

In addition to providing more information 
directly related to the IPD measures in the 
Goals Report for the year 1995, the 
suggestions in other items would provide 
additional measures on the following topics 
for the goals report for the year 2000: 

Item 3: Planning and Coordination of IPD. 
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Item 4: Why Teachers Choose Not to 
Participate in IPD Programs. 

Item 5: School Organization for Teacher 
Learning and Other Growth Opportunities 
Provided for Teachers. 

Item 6: Incentives to Participate in IPD. 

Item 7: Creating Time for Professional 
Development. 

The selection of indicators for the 1995 
Goals Report was limited by data 
availability. The resource group identified 
some future data needs, two of which 
might be filled by SASS 1998-1999: 

The matter of equity regarding the 
number of teachers having full 
access to high-quality professional 
development activities should be 
reported. 

Partnerships that provide and 
support programs for the 
professional development of 
educators should be established. 

Data for both of these topics could be 
provided by adding questions to the 
Principal’s Questionnaire. As proposed, 
Item 3 on Planning and Coordination 
contains the question: "Is your school or 
school district engaged in partnerships that 
will promote community stakeholders’ 
support of programs for professional 
development of educators? Item 5 on 
School Organization for Teacher Learning 
and Other Growth Opportunities Provided 
for Teachers contains the question: "Do 
all teachers have full and equal access to 
high-quality IPD activities? 



Board certification. The Resource Group 
also suggested indicators that show the 
number of teachers who are seeking to 
become board certified and the number of 
school districts that are supporting teachers 
who are seeking National Board 
certification. Information on the number 
of schools that are supporting teachers who 
are seeking board certification could be 
obtained from Item 5 which includes the 
question "Does your school district support 
teachers who are seeking National Board 
certification?" Information on teachers who 
are seeking or who have received National 
Board certification could be obtained by 
adding questions to the Teacher’s 
Questionnaire in the section on teacher 
training. 

Item 12: 14 Board Certification. Add a 
new question with the two parts: Have 
you received National Board certification? 
Are you seeking National Board 
certification? 

In summary, the Year 2000 Goals Report 
could provide a much more comprehensive 
picture of progress toward Goal 4 if the 12 
suggested items were adopted. 



Summary 

How well do the suggested items respond 
to the 14 introductory questions? Early in 
the paper it was noted that it is not feasible 
for SASS to collect the data required for 
Questions 6 and 10. However, 
information about some aspects of these 
questions would be provided by other 
suggested items. Teachers’ perceptions of 
the effectiveness of the IPD programs 
would be known and certainly Item 1 on 
prevalence of IPD by type and Item 2 on 
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program content and length would provide 
a good idea of what the public sector 
investment (whatever the amount may be) 
is purchasing. Question 9 will be covered 
in a later paper. Excluding these three 
questions, items have been suggested that 
provide data that relate directly to all the 
remaining questions except the last four. 
One of these, number 13, is addressed in 
part in Part II. Analysis of the data from 
the suggested items can also provide some 
information relevant to better ways to 
invest resources (Question 11). 
Documentation of the range and quality of 
IPD in 1998-99 can also be the basis for 
suggesting the changes needed in IPD to 
meet the challenges of the reform 
movement (Question 14). Finally, it 
should be possible to measure how the 
characteristics of IPD are changing over 
time (Question 12), since care was taken 
to preserve the options in the 1993-1994 
SASS when modifications of questions 
were suggested. 

Including all 12 suggested items in SASS 
would greatly expand the section devoted 
to IPD. It is important, however, to 
remember the essential role of the teacher 
in the reform effort and the importance of 
providing teachers with the degree of 
professionalization needed in reform. We 
should not forget Sykes’ assertion that the 
most serious unsolved problems for 
American education today are that teacher 
learning must be the heart of any effort to 
improve education and that conventional 
professional development is sorely 
inadequate (Sykes, 1996). Policymakers 
need information to address these 
problems. It follows that inservice 
professional development should be given 
the attention and the space in the 1998- 
1999 SASS that it deserves. 



PART II 

International Comparisons of IPD 
for Use of Computers and 
Advanced Telecommunications 
Equipment 

Part II discusses the value of international 
comparisons and the value of state and 
nation comparisons generally and more 
specifically with respect to IPD. A 
number of international comparative 
studies that have been reported in process 
or in the design stage will provide data on 
IPD and related topics such as school 
organization and environment. 15 Part II, 
however, addresses only one international 
study, the IEA Computers in Education 
Study (CompEd Study), which has 
extensive information on the professional 
development of teachers. It is discussed 
here to allow ample time for careful 
evaluation of the suggestion made in a 
later section to incorporate items from the 
CompEd Study in the 1998-1999 SASS. If 
implemented, this suggestion would have a 
large impact on SASS and should be 
considered in the early stages of the 
development of SASS. Part II therefore 
continues with a discussion of the need for 
data on IPD for use of computers and 
advanced telecommunications equipment, a 
specific proposal to include IPD items 
from the CompEd Study in SASS, and a 
description of the benefits of doing so. 

Value of International 
Comparisons 

The SASS measurements could be made 
more meaningful and the policy-relevance 
of the data could be enhanced by 
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comparisons with data from other nations. 
A number of educators have discussed the 
value of international comparative 
education information. Bradburn and 
Gilford (1990) consider that the most 
important use is to improve understanding 
of our own education system. In the 
absence of absolute standards for 
educational systems, comparative 
information can contribute to setting 
realistic standards and to monitoring the 
success of educational systems. They note 
the value of comparisons with other states 
or the nation, comparisons that have the 
advantage of comparing systems that are 
broadly similar. International 
comparisons, however, expand the range 
of comparison beyond the limits of 
national experience, and can be helpful not 
only for descriptive purposes but also for 
monitoring. Plomp (1992) also considers 
that the most important reason for 
international comparisons is to improve 
understanding of educational systems and 
to provide policymakers and educators 
with information about the range of 
educational quality among various national 
systems. Cross and Stempel (1995) note 
that the value of international information 
is that it provides the opportunity to 
resolve the failings of our system in a 
uniquely American way. They urge 
concentrating on the reasons behind the 
decisions made by different countries 
concerning teacher training policies. 
Understanding their motives and 
expectations will help us decide what will 
and will not help us improve primary and 
secondary education in America. 



Similarities in Cross National Issues 

Most developed countries are facing 
similar education policy issues. Several 
countries are involved in reform efforts 
and many are faced with the issue of how 
to provide high-quality education to a 
multicultural student body. These 
common concerns enhance the likelihood 
that we can learn from the actions taken by 
other countries. Some of the issues relate 
specifically to IPD. Most countries that 
are members of the OECD are deepening 
inservice teaching opportunities, as are 
Asian countries, e.g., Japan, Taiwan, and 
China (Darling-Hammond, 1996). There 
is concern about the limited opportunities 
for advancement and promotion in 
teaching. To address this issue, some 
countries, e.g., the United States and New 
Brunswick, Canada, are taking action to 
create a career path that would lead toward 
highly accomplished practice over the 
course of a teaching career. Korea has 
also recognized the need for a teaching- 
oriented career continuum (Darling- 
Hammond & Cobb, 1995). In Spain and 
Portugal inservice training is linked to 
career advancement (EURYDICE, 1995). 

Another issue common to several countries 
stems from recognizing the importance of 
giving teachers greater professional 
authority and responsibility. The United 
States, Manitoba and Quebec in Canada, 
and the Republic of Korea have responded 
by giving teachers greater professional 
autonomy and greater voice in creating 
standards for preparation, licensure, and 
practice. Several European and Asian 
countries have recognized the significant 
role of continuous professional 
development as an important part of 
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professionalism (EURYDICE, 1995; 
Darling-Hammond & Cobb, 1995). 

Value of State and Nation 
Comparisons 

The policy relevance of the SASS data 
could be further enhanced by comparisons 
of state and nation data. SASS certainly 
has the potential to provide state data on 
IPD. Although the SASS by State 
publication (NCES, 1994) includes data 
about teacher characteristics and their 
preservice preparation, it does not include 
IPD data. During the current period of 
extensive reform and restructuring of the 
schools it is important for states to know 
about the involvement of the current 
teaching staff in the reform effort and how 
teachers are upgrading their expertise in 
their field and in pedagogy to meet the 
demands of reform. If the suggestions in 
this paper are implemented, it would be 
possible to provide state data on the types 
and extent of IPD activities, the planning 
and coordination of IPD, school 
organization for teacher learning and other 
growth opportunities provided to teachers, 
support for IPD, and the school 
environment. It should also be possible 
for states to compare some characteristics 
of their professional development activities 
with those in other countries in a format 
like that used in the NCES publication, 
Education in States and Nations (1996). 
Because of the central role that teachers 
play in student achievement, states that 
have demonstrated interest in educational 
achievement in other countries (frequently 
for economic reasons) would find uses for 
such information. 



We turn now to ways of obtaining IPD 
data for state and nation comparisons and 
international comparisons for one topic: 
use of computers and advanced 
telecommunications equipment. We first 
consider why such data are important at 
this time. 

Need for Data on IPD for Use of 
Computers and Advanced 
Telecommunications Equipment 

This section describes the rapid growth in 
the use and types of use of computers in 
the schools, and the political support for 
introduction of advanced 
telecommunications equipment in the 
schools. Although IPD in the use of 
computers and other technologies is an 
important aspect of the successful 
introduction of the equipment in the 
schools, little is known about it. There 
are large gaps in the U.S. system of 
teacher training: teachers need more time 
to become conversant with computer 
technology, to plan lessons that integrate 
the computer in classroom activities, and 
to learn about computers (Anderson, 

1993). The Comp Ed Study is of special 
interest because of its careful look at IPD 
for use of computers and because of its 
finding that U.S. teachers had less 
opportunity for such IPD than teachers in 
countries whose students were more 
proficient than U.S. students in the use of 
computers. This section concludes with a 
description of the IPD items in the 
CompEd Study. 
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The Use of Computers in Education 

As we move into the age of cyberspace, 
there are many unanswered questions 
about the role that computers and other 
forms of technology can play in education. 
The percentage of elementary and 
secondary school students who use a 
computer at school is increasing rapidly: 
in the nine years from 1984 to 1993 the 
percentage doubled, increasing from 28.5 
percent to 59.0 percent (NCES, 1996). 
Fulton (1996) estimates that there were 
almost 5 million computers for 
instructional use in K-12 schools in 1995 
and that the expenditures on technology 
reached $2.4 billion a year. Policymakers 
are justly concerned about the effectiveness 
of an investment of this size; they need 
additional data about computers, how they 
are used in the schools, and how they 
improve teaching and learning. 

Changes in the Use of Computers in the 
Schools 

Changes are rapid in this field. Recent 
new releases (Washington Post, 1996) 
illustrate two such changes. The first 
describes an experimental program in 
Germantown, Maryland, using the 
computer as an online algebra instructor to 
replace a human teacher and a program in 
an Alexandria, Virginia, school where 
fourth- and fifth-graders can choose to 
learn math from a computer or a teacher. 
The second announces the "Net Day" on 
March 9, 1996, when most of California’s 
13,000 public and private schools were 
scheduled to be wired for the Internet. 
Television on that day showed both the 
President and the Vice-President 
participating in the wiring! The National 
Information Infrastructure proposed by 



President Clinton includes a goal to 
connect all the nation’s school classrooms 
(and also various other institutions and 
organizations) to the "Information 
Superhighway." It remains to be seen how 
teachers will use access to the Internet in 
their teaching. 

Importance of IPD in the Use off Computers 
and Advanced Telecommunications 
Equipment 

Teacher training is an important aspect of 
the introduction of computers in schools 
because most of today’s teachers did not 
use computers when they were in 
elementary and secondary school and many 
of them did not receive computer 
education as part of their preservice 
training. On the other hand, it is noted by 
Pelgrum and Plomp (1993) that: 

"...teachers are ultimately the ones 
charged with the implementation of 
computers in educational practice and 
therefore education of the educators or 
teacher training is an important aspect of 
the introduction of computers in schools." 
In fact, most of the benefits students will 
derive from using computers depend on 
the extent to which teachers integrate 
computers in their daily classroom 
activities. In 1992, however, less than 
half the schools in the United States 
reported having an introductory computer 
course available for teachers (Anderson, 
1993, p. 52). American teachers have less 
opportunity to take inservice computer 
courses than do teachers in Austria, 
Germany, and the Netherlands and, as 
might be expected, students in these 
countries are more computer- 
knowledgeable than American students 
(Anderson, 1993). 
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More recently, a survey to obtain baseline 
data on the status of advanced 
telecommunications in public elementary 
and secondary schools asked about barriers 
to the school’s acquisition of advanced 
telecommunication capabilities. Nearly 
two-thirds of the surveyed schools cited 
lack of or inadequately trained staff and 
lack of teacher awareness regarding ways 
to integrate telecommunications equipment 
into curricula as moderate or major 
barriers (NCES, 1995). 

IPD Items in the CompEd Study 

The aim of the first stage of the study, 
with data collection in 1989 was "to obtain 
information about the current status of the 
use of computers in education, more 
specifically within schools, ...for use in 
planning, implementation and evaluation in 
the field of computers in education" and to 
provide baseline information for measuring 
change in stage 2, with data collection in 
1992. In addition to obtaining data to 
measure change, "...stage 2 involved 
assessing effects of school variables, and 
teacher and teaching variables on student 
outcomes in the domain of computer usage 
in schools (functional computer knowledge 
and skills)" (Pelgrum & Plomp, 1994). 

The survey included questionnaires for 
principals, school computer coordinators, 
and teachers of mathematics, science, 
mother tongue, and computer education in 
the grades 5, 8, and 11. 

The CompEd Study includes a number of 
questions about teachers’ professional 
development related to the implementation 
of computers in educational practice: 



A self-rating scale about the 
teacher’s knowledge about and skill 
level in using computers 

Problems experienced in using 
computers. The list of problems 
includes three that are related to 
professional development: (1) 
teachers lack knowledge/skills 
about using computers for 
instructional purposes, (2) 
insufficient training opportunities 
for teachers, and (3) lack of 
interest/ willingness of teachers in 
using computers 

Teacher’s opinion of training needs 

Training received-provides a list of 
25 topics covered in training. 
Information on teacher training is 
important because the CompEd 
Study found that teachers tend to 
teach the topics covered in their 
own training in the lessons for their 
students. 

Support for training: availability of 
training at school; agencies that 
provide training support; 
availability of and type (full time, 
teacher, etc.) of computer 
coordinator in the school; time the 
computer coordinators spend 
helping teachers use computers (in- 
school support) or in training or 
study for themselves 

Percent of teachers using computers 
in mathematics, science, English, 
or computer education 
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The following section makes a proposal 
for using some of the CompEd IPD items 
(augmented by items related to advanced 
telecommunication) in the 1998-1999 
SASS and discusses the benefits of doing 
so. 

Proposal to Measure IPD for 
Computer Education in the 1998- 
1999 SASS 

It is proposed that SASS include IPD 
questions from the CompEd Study. This 
would require additional questions in the 
Principal Questionnaire, a new Computer 
Coordinator Questionnaire, and either 
additional questions on the Teacher 
Questionnaire or a separate questionnaire 
for a sample of teachers in the fifth, 
eighth, and eleventh grades. 

Although the CompEd Study did not 
address the IPD aspects of teachers’ use of 
advanced telecommunications equipment, it 
would be useful to include questions in the 
Teacher Questionnaire on how teachers use 
networking and other forms of advanced 
telecommunication, the training teachers 
receive to prepare them to use technologies 
as teaching tools and resources, and their 
awareness of the resources technology can 
offer them as professionals in carrying out 
many of the activities of their jobs (Fulton, 
1996). Fulton develops this topic more 
extensively. A focus group to address 
ways to incorporate such questions in the 
survey without losing comparability with 
the data from the CompEd Study could be 
useful. 

The magnitude of the impact of this 
proposal on SASS is recognized. To 
compensate for the response burden it 



would create, it could replace the Teacher 
Demand and Shortage Questionnaire in the 
1993-94 SASS. The most essential 
questions from that survey could be added 
to the Principal Questionnaire. 

Benefits of Including IPD Items from 
CompEd in the 1998-1999 SASS 

There are several reasons why it would be 
useful for the 1998-1999 SASS to include 
some of the CompEd stage 2 questions 
about inservice development of teachers. 
First, because many types of experts were 
involved in developing the IEA survey, it 
has led not only to interesting findings 
about the status of professional 
development for computer education, and 
identification of large differences between 
countries in IPD, but has also provided 
data useful to policymakers. For example, 
data from the CompEd Study (a) provided 
the basis for recommendations concerning 
the training needs of teachers, (b) made it 
possible to identify the relative position of 
a country with respect to the availability of 
training and support for teachers, (c) 
provided a measure of the extent to which 
the computer was integrated in classroom 
teaching, and (d) made it possible to 
determine the relationship of teacher 
training to actual classroom use of 
computers. Second, including CompEd 
IPD questions in SASS 1998-1999 would 
make it possible to measure change in the 
amount and character of computer IPD in 
the United States from 1992 to 1998. 

Third, it would permit states to compare 
their IPD in 1998-1999 with that of other 
states and the nation. And fourth, 
although there would be six years 
difference in the data, states could 
compare IPD for their teachers with that of 
teachers in other nations at an earlier time. 
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In summary, the importance of this 
proposal is supported by the combination 
of rapid growth in the use of computers 
and advanced telecommunications 
technology in the schools, the essential 
role that teachers play in their effective 
use, the inadequate training in their use 
that is available to teachers, and the 
national will for U.S. students to match 
the achievement of students in other 
countries in the use of these technologies. 
Our national leaders have already 
recognized and recently underscored the 
importance of such technologies in 
education. 
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Mandel (1995) is the source for Questions 4, 7, and 10. 

Because networks have become an increasingly important form of IPD in the reform of U.S. education 
Lieberman and Grol nick (1996) studied 16 networks to ascertain how they are formed, their focus, and how 
they are sustained. They note that networks provide ways of learning that are more in keeping with the 
professional lives of teachers. Networks engage " . . .school-based educators in directing their own learning, 
allowing them to side-step the limitations of institutional roles, hierarchies and geographic locations, and 

encouraging them to work together with many different kinds of people. " 

Provides information related to introductory Questions 3, 4, 7, and 12. 

Provides information related to introductory Questions 4 and 5. 

Provides information related to introductory Question 1. 

Provides information related to introductory Question 8. 

Provides information related to introductory Question 8. 

Provides information related to introductory Question 8. 

Provides information related to introductory Question 8. 

Provides information related to introductory Question 8. 

Provides information related to introductory Question 2. 

Provides information related to introductory Question 2. 

Provides information related to introductory Question 2. 

Provides information related to introductory Questions 3, 4, and 8. 

The potential of all of these studies as a source of IPD-related items for the 1998-1999 SASS will be 
explored in a later paper. The paper will also provide analysis plans describing how data from these 
studies might be used in international comparisons or in state and nation comparisons of IPD— if the 
1998-99 SASS includes comparable items. 



DISTRICT LEVEL DATA IN THE SCHOOLS AND STAFFING SURVEY 



J. Michael Ross, National Center for Education Statistics 



The importance of district-level data in 
systematic assessments of changes in the 
organizational structure of schools as 
educational institutions is increasing. 1 
This raises the question of whether the 
next Schools and Staffing Survey (SASS) 
should shift its focus toward district-level 
processes rather than toward classroom 
instructional models as has been proposed 
in other papers in this seminar series. 2 
The arguments for a reconsideration of the 
role of district-level data in SASS are 
derived from several sources: (1) a review 
of sociological theories as applied to the 
organization of education in the United 
States; (2) a critical review of the 1993 
SASS district-level survey and its ability to 
uniquely answer important research 
questions; (3) the increasing importance of 
“choice” mechanisms for student 
assignment policies in public schools; 3 and 
(4) recent policy research based upon 
studies of state-wide systemic reform 
efforts. 

Many of the important organizational 
issues outlined below are amenable to 
more systematic empirical exploration even 
with the 1993 SASS (particularly as they 
involve between-state and between-district 
variations). The argument presented here 
is that the local school district is still an 
important mediating organization in the 
implementation of educational policy. 

From this review, it should be apparent 
that the 1993 SASS district-level survey 
should be supplemented by more yearly 
Common Core of Data (CCD) school 
enrollment information (including 1990 



demographic data from the School District 
Data Book), staffing data, and fiscal data. 
Through an examination of data on magnet 
schools, the feasibility of a multi-level 
linked approach will be examined in the 
context of a different sampling strategy. 
For some, the need for more 
programmatic information that can be 
provided by district- and school-level 
administrators is still an open-ended 
question for 1998. For others, however, 
top-down models of analysis are the 
prevailing, if not the only, strategy to 
study reform implementation effects. 
Research studies suggest that the addition 
of critical reform data at the district level 
could enable SASS surveys to become the 
established baseline survey for a large 
variety of hierarchical studies by the U.S. 
Department of Education and the National 
Science Foundation. 

Administrative Structures and District 
Organization 

Long-term historical trends . The state role 
in education has continued to expand 
rather than diminish during the 1990s. 
State funding and programmatic control of 
education has complex organizational 
implications for school districts and the 
management of schools within these 
districts. 4 Through the 1980s 
relationships among different levels of 
government (federal, state, and local), 
became increasingly complex, and at the 
same time, a more layered, formalized 
structure of control (multi-level and 
centralized) continued to develop. As the 
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external environment imposed a 
multiplicity of new requirements, 
administrative complexity expanded 
substantially at the intermediate (i.e., 
school-district) level. 5 

A major factor in this increased 
complexity has been an increase in 
categorical funding at the federal level for 
a large number of special programs, and 
the emergence of new administrative 
subunits to monitor and oversee these 
programs within states and local school 
districts. 6 These additional layers of new 
authority typically have not displaced 
existing structures, and the current system 
of governance has preserved the legal 
autonomy of lower levels of power, 
primarily local school districts. 

Local districts are 
fundamental governance 
agencies, by tradition and 
practice .... and their 
influence is extraordinary in 
world perspective. Despite 
the recent growth of state 
and national power, these 
districts make a great range 
of decisions, including those 
that bear on levels of 
funding, the nature of 
educational programs, and 
the teachers to be hired. 

(Cohen & Spillane, 1992) 

By most accounts, the resulting structures 
are highly fragmented, and from more 
critical perspectives “incoherent” (Cohen, 
1995). 7 The difficult (or unsettled) 
question is whether local responses to 
these new reform initiatives have amplified 
differences (i.e., increased rather than 
decreased variability) between districts and 



schools as new instructional policies are 
filtered through fragmented and 
heterogeneous organizations. 

The diminishing role of local revenue in 
the operation of local schools means these 
authorities are now held more accountable 
to standards imposed by external funding 
sources and to the parallel need to 
centralize the budgeting process and 
personnel decisions at the district level. 8 
The resulting interdependence between the 
district and its component schools has 
required more administrative coordination 
and an increasing number of professional 
administrators to “manage” the schools. 
One critical element of this administrative 
growth has been the addition of categorical 
programs (and the external accountability 
requirements) from state and federal 
sources to each district’s operating budget. 
At the individual school level in larger 
school districts, the administration of these 
programs has involved a parallel increase 
in the number of administrators and 
program specialists. In some schools, 
however, the bureaucratic burden of many 
separate programs has generated a variety 
of school- wide reforms, and, consequently, 
ongoing decentralization efforts have been 
designed to counter the organizational 
effects imposed by the demands from these 
external authorities. 

Analytic Role for SASS District-Level 
Data 

The primary rationale for the district-level 
survey (still identified as Teacher Demand 
and Shortage Questionnaire-TDSQ) in its 
first administration in 1987 was national 
concern over the prospects of teacher 
shortages, particularly in specialized fields 
and special programs. 9 Before a more 
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detailed review of other issues related to 
district-level information is undertaken, it 
is important to identify the main questions 
that were the primary focus of the Local 
Education Agency (LEA) Questionnaire, 
and how analysis of administrator and 
teacher surveys provided alternative 
strategies to satisfactorily answer these 
questions. 10 In the 1987 and 1990 SASS, 
the “number of positions filled” was 
consistently high, approximately 99 
percent in both years. As reported by 
district administrators, less than 0.5 
percent were vacant or unfilled. Similarly, 
districts reported that nearly 10 percent of 
their teachers were new hires, indicating 
that when positions became available they 
could find qualified teachers from 
available sources of new college graduates, 
teachers in other districts, private schools, 
or other sources. 

At the other end of the spectrum, districts 
provided counts on how many teachers had 
been “laid off” for budgetary reasons (i.e. , 
RIFs). In 1990, the percentage was only 
0.6 percent. Although the percentage of 
“laid-off” teachers is not included in the 
1993-94 SASS Statistical Profile (because 
previous year estimates of faculty were not 
asked), a slightly different calculation for 
districts with more than 100 teachers 
indicated that 162 districts had reductions 
greater than 5 percent. Included in this 
group are several well-known districts 
where these large cuts merited notice in 
local newspapers (and even in Education 
Week). Clearly, the down-side in teacher 
staffing numbers is concentrated in one 
year when a budgetary crisis (often 
precipitated by declining student 
enrollments) occurs. As the frequency of 
SASS shifts to every five years, the 
inclusion of faculty trends for prior years 



from CCD may be more necessary to 
identify these long-term trends. 

Only a few tables in the SASS Statistical 
Profiles contain data from the TDSQ. 

Many of the demand issues are adequately 
addressed by results at the school level 
(Table 7.2, Statistical Profiles: 1990)" 
and by questions related to the relative 
difficulty in filling vacancies by specific 
fields (Table 7.3 and Table 7.4). 

Likewise, the analysis of school-level 
turnover rates (as measured by the 
percentage of teachers who left positions in 
the school in the prior 12 months) allows 
statistical analysis of school characteristics, 
private school status, district-level salary 
and benefits, and even workplace climate 
(Ingersoll, 1996). In regard to the 
availability of new teachers and other 
characteristics of the teaching profession, 
the individual teacher survey has provided 
more detailed information on new teachers 
(Rollefson & Broughman, 1995). 12 And 
finally, the teacher followup survey 
provides even more detail on the flow (and 
the reasons) that teachers move to other 
positions or leave the profession (Bobbitt 
et al., 1991, Bobbitt et al., 1994). 

Districts after schools . Schools are 
sampled first, and districts are included if 
one of their schools is selected. As a 
result, the average district in 1993 was 
represented by 1.7 schools. When national 
and state-level estimates are made for 
student enrollment and staffing data, there 
is substantial overlap, and consequently the 
school-level information and the district 
data are redundant. 13 Incorporating some 
CCD district data directly into the 
interview instrument (but taking into 
consideration a lag between sample 
selection of schools 14 and data collection 
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time frames), and then the refinement of 
this information through more structured 
survey questions would be a major design 
improvement. In addition, basic data could 
be added on enrollment and teacher data 
for a fixed number of prior years. 15 The 
district-level data should also include 
aggregated counts (students and teachers) 
for all schools in the district. 

In the process of linking schools to 
districts (and both units back to teachers), 
certain varieties of governance structures 
become evident. Some of these state- 
specific categories are more accurately 
identified in the current CCD district 
classification system. 16 District and 
school eligibility criteria should be 
reviewed in order to consider other types 
of instructional and support staff counts 
that are included in the agency universe 
survey since in some districts they are 
becoming more important elements in the 
“reform agenda.” 

In reconsidering the utility of the district 
survey, the functions administrators 
perform should be reviewed in order to 
consider whether these administrative 
responsibilities should be incorporated into 
the next SASS. District staffs historically 
have had limited authority for instruction 
conducted in the classroom by teachers. 
Nevertheless, a high percentage of 
expenditures are no longer associated with 
instructional staff as conventionally 
defined. 17 A large number of routine 
administrative and budgetary tasks (some 
are generally not relevant to the objectives 
of SASS) are still performed by district 
staff. Some information, such as starting 
teacher compensation with different 
degrees and benefits (Table 5.3 and Table 
5.4, 1990 SASS Statistical Profile) are 



already core variables and other questions 
have been asked for two administrations of 
SASS (Levine & Christenson, 
forthcoming). A large number of 
personnel and student assignment decisions 
are made by district-level administrators 
(public schools only). But, the personnel 
office performs many other critical 
functions for teachers working in a 
district. Teachers are typically hired by the 
district and then assigned to specific 
schools. 18 Likewise, between-school 
transfers of students and teachers are 
required to adjust for shifts in student 
populations, and periodically school 
openings and closings necessitate even 
larger adjustments. More process 
questions seem to elicit more useful 
information about the outcomes of 
administrative decisions (e.g., the school 
questionnaire could ask how vacancies 
were filled in a school with a check list 
and a rating of difficulty). District 
administrators could be asked how they 
have recruited new teachers over the last 
few years (types of strategies such as 
visiting local college campuses, national or 
local advertising), in what disciplines 
teachers were hardest to find and then hire 
teachers for (at this point pay incentives 
would be relevant to ask). Certification 
requirements for new teachers are usually 
established at the state level, and district 
administrators can offer more information 
about recent changes in these policies. 19 

New policies for “student performance” 
(e.g., the number of courses required for 
graduation and more rigorous standardized 
tests) have been enacted in recent years by 
many, but not all, states. The district 
questionnaire could ask whether a change 
has occurred (there should be a high 
degree of consistency within states) and 
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then the respondent would indicate how 
the number of course credits changed 
when the policy was implemented. 
Likewise, most large districts have some 
written discipline and substance abuse 
policies (primarily for legal reasons). In 
response to federal and state initiatives, 
new policies have been adopted, and the 
discipline implications for similar 
infractions represent a new policy 
dimension. 

Many important policy decisions are based 
on school board actions, state legislation, 
and new federal programs that often are 
not best analyzed at the individual school 
level. 20 For many programs, the funds 
are identified separately in terms of dollar 
amounts and funding sources. For 
example, the Eisenhower professional 
development grants are administered by 
local districts after they apply to state 
departments for approval of programs. 
Individual districts have wide latitude in 
the use of these funds and may use them 
for such activities as professional 
workshops at a local university, national 
conferences, or instructional sessions for 
teachers within a district. 21 

Even more relevant are student assignment 
issues that are a policy realm under the 
nearly exclusive control of school district 
authorities rather than individual schools 
(with the exception of some districts still 
under federal desegregation court orders). 
In a narrow historical perspective, the 
district activities in this realm were quite 
conventional: fixing physical boundaries 
(that rarely changed), constructing a new 
school when enrollment expanded rapidly, 
selecting which school to close when 
enrollments declined, and then deciding 
which schools to consolidate (with limited 



adjustments in surrounding schools). In 
physically large districts, transportation 
imposed another set of fiscal and resource 
constraints. In the last 25 years, 
fundamental change has slowly displaced 
“the neighborhood school” linked solely to 
residence. In large central city districts, 
the change was abrupt when federal 
desegregation plans imposed new 
geographic configurations, but the 
transition was also facilitated by 
experimentation with district-wide “magnet 
schools” based upon distinct instructional 
programs that would attract different-race 
students. Besides the obvious benefit of 
dismantling “racially identifiable” schools, 
magnet schools enabled some schools 
within a district to formulate their own 
content emphasis, special themes, or 
school philosophy (and also recruit their 
own faculties for these purposes). The 
traditional uniformity of schools imposed 
by a central “bureaucracy” no longer 
maintained its total control over students, 
faculty, and instruction in these schools, 
and for the first time “market 
mechanisms” were incorporated in the 
school selection process (parents had an 
option to choose a magnet for their 
children or could leave if they were not 
satisfied). At this point, it is evident that 
identification of specific magnet schools 
can only be obtained at the district level 
where student assignment policies are 
implemented. 

Multi-level analysis . The utilization of 
district-level information in prior SASS 
surveys and reports has been quite limited, 
and the additional questions included for 
the first time in 1993 (AIR, 1996) 
probably will not change interest in 
complex multi-level analysis. Ingersoll’s 
An Agenda for Research on Teachers and 
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Schools (1995) does not identify a single 
issue where district-level data are a 
decisive factor in an important research 
question. Only the recent studies by 
Chambers (1995 and 1996) explore the 
differential effects of school and district 
characteristics (i.e., measured by cost 
factors that local decision makers cannot 
control) on teacher salaries. 22 For 
example, the relationship between salaries 
and the racial composition of the district 
showed that only the percentage of 
students who were Asian Americans had a 
significant effect (the school-level analysis 
had significant effects only for percentage 
of Hispanic). In the Chambers study, 
however, the inclusion of district-level 
(and school variables) resulted in a 
substantial loss of schools (17.1 percent) 
and also teachers in the sample. The 
Ingersoll (1996) analysis also had a large 
erosion in its school sample size when 
such district-level variables as availability 
of merit pay plan, paid benefits, and 
district size were included in his analysis 
(and most not significantly) of net teacher 
turnover rates. 23 

The Chambers study (1996), although 
based upon 1990-91 SASS, does at least 
provide a model for making decisions 
about which type of questions should be 
asked to whom based upon multi-level 
statistical analysis. In his analysis, only a 
few district-level variables have a 
significant effect in explaining differential 
teacher salaries. In fact, the three district- 
level variables in his regression equations 
(district size, racial composition of the 
district, and enrollment growth) were 
obtained from CCD, and it is reasonable to 
assume that data from the district survey 
on pay incentives or fringe benefits 
probably would not explain additional 



variation in teacher salaries (above and 
beyond the combined effects of individual 
teacher background factors and school- 
level factors). Using this method, one 
could determine if there was a district- 
specific effect for pay incentives offered to 
mathematics teachers for example, 
controlling for their education background 
and years of teaching experience. 

Likewise, this method of multiple -level 
analysis could determine the additional 
contribution of district policies to 
differential teacher salaries. 24 

Multi-level analysis is complex, and this 
alone might account for the limited use of 
district- level data by researchers. 

Similarly, missing data problems 
unexpectedly escalate when a district 
nonresponse for a large district eliminates 
several schools from the sample. 

Moreover, the original CCD identification 
is difficult to reconstruct “after the fact” 
for schools with a missing district survey. 
Most statistical software packages do not 
allow other sources of district data to be 
easily incorporated after SASS analysis 
files have been merged. Finally, multi- 
level analysis of between-school 
differences (controlling for district context) 
is severely limited by the nonhierarchical 
design feature of the SASS sampling 
strategy given the small percentage of 
districts with more than one school per 
district. 

Schools after districts . The limited 
number of key variables in the SASS 
district-level questionnaire imposes 
practical difficulties in linking different 
levels. How conceptual issues related to 
district policies in turn impact schools 
within each district suggests a different 
design strategy for the new SASS: 
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sampling districts first (based upon the 
number of teachers in the district’s 
schools), and then sampling schools within 
these selected districts. A larger number of 
schools per district 25 thereby would be 
sampled in districts that have more than 

10.000 students, for example. In 
Appendix I, a comparison between the 
average number of schools per district 
with a district sampling strategy is 
presented with the results of the 1993 
SASS. 

Take a state such as Florida with large 
county-wide districts. SASS 1993 samples 
258 schools, but they are scattered across 
55 districts, giving an average of only 4.7 
schools per district. 26 With districts 
sampled first, only 20 districts would be 
selected with an average of 20 schools per 
district. Utah, a more typical state, would 
have an average of 12.9 schools per 
district drawn from a sample of 20 
districts (in 1993, the SASS average is 5.5 
schools per district). Obviously in rural 
states, such as Iowa, an average of 2.8 
schools per district is not a substantial 
improvement over 1.3 schools per district, 
and fewer sampled districts (67 versus 128 
in 1993 SASS) does not improve district 
estimates when most districts are quite 
small. There are more than 175 districts 
that have student populations greater than 

25.000 students (most have more than 25 
schools) and at least five to ten schools 
from each district would be selected with 
this strategy. 27 The number of schools, 
however, is also a function of the relative 
concentration of students in larger 
districts, and the average number of 
schools would vary by state. 28 

Furthermore, a district-level survey would 
allow direct links to individual schools in 



each district through the LEA 
questionnaire. Federal and state program 
funds are allocated to specific schools 
within a district, and accordingly the 
number of instructional staff allocated to 
these programs (such as magnet schools) 
would be enhanced through this type of 
multi-level design. 

District-wide Assignment and Choice 

Districts have developed schools with 
special academic programs that attract 
students on a district-wide basis in order to 
comply with federal desegregation court 
orders. These “magnet” schools first 
emerged in the late 1970s in several large 
northern districts. Initially, they often 
were part of a more global restructuring of 
school attendance boundaries and the 
emergence of noncontiguous assignment 
policies that have been responsible for 
major revisions in conventional 
“neighborhood” attendance zone practices 
in many districts. Their growth has been 
accelerated by federal grant programs 
(Emergency School Aid Act, Magnet 
School Assistance Program- 1983), and 
increasing acceptance of voluntary student 
assignment components in desegregation 
plans by federal courts in the mid-1980s. 

In the last five years, more districts have 
been released from court supervision and 
have adopted expanded student choice 
options as a replacement for mandatory 
policies. Other reform movements have 
stimulated a broad interest in specialized 
“choice” schools in districts (often 
encompassing an entire district with 
“racial balance” a minimal consideration). 

The particular combination of conventional 
attendance zone and district-wide choice 
policies reflect a district’s long 
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desegregation history. In broad terms, one 
needs to know when a district first 
desegregated its schools; whether a partial 
or district- wide remedy was required; 
whether a plan was phased in over time; 
the statistical guidelines used for these 
desegregation plans; and the racial 
composition (and size) of the district when 
the first substantial desegregation plan was 
implemented. 29 Certain historical 
parameters, although difficult to establish 
initially, facilitate the tracking of these 
periodic modifications that are used by 
many large districts to maintain certain 
levels of “racial balance” through the 
provision of choice policies. 30 The 
evolution of district plans also shows the 
variations in magnet schools, from the 
“ideal” district- wide option for both 
minority and majority students, to schools 
that have district-wide options for students 
of one race and neighborhood options for 
students of another race, and to smaller 
programs-within-schools where district- 
wide options are mixed with neighborhood 
assignment policies. In most cases, only 
the recent history is relevant since large 
changes in student assignments require 
well-publicized announcements by district 
administrators. 

Most large districts are expanding choice 
elements in many schools and the 
implementation of more specialized 
curriculums has an indirect effect on those 
schools that do not offer “new” programs. 
Students are selectively drawn from large 
geographic areas and some teachers are 
“redeployed” from other schools to staff 
these new programs. Rarely has a magnet 
school been opened without consideration 
of district-wide racial balance (either to 
attract other-race students for the first time 
or to maintain desegregation when a 



district shifts from mandatory to voluntary 
assignments for some students). Even one 
magnet school with district-wide 
enrollment options has ah impact on which 
students are enrolled in other schools. 
These attendance policies are administered 
by district administrators and school-level 
principals are not fully aware of the 
interrelationship between who attends their 
school and the schools other students 
attend as a consequence. 

Choice in the 1993 SASS . Questions 
concerning magnet schools were asked in 
both the SASS district-level and school- 
level questionnaires, and a reanalysis of 
the data offers a preliminary view of the 
inherent difficulties in translating policy 
options into clean simple survey language. 
Alternative question formats and more 
reliable results are fortunately available 
from a survey conducted in 1991-92 by 
AIR on magnet schools and desegregation 
plans. The 1993 SASS district-level 
survey asked whether students could 
“enroll in another school or district outside 
their attendance areas” 31 and if the 
answer was “yes,” the respondent could 
check enrollment in a magnet school or 
“enrollment in any school in this district” 
(and then the respondent would estimate 
the number of students in each program). 
Approximately 579 districts indicated that 
some students attended magnet schools. 32 
It would have been preferable if the SASS 
questionnaire had provided a definition of 
“choice” that included some reference to a 
special emphasis or distinct curricular 
theme, and a district-wide enrollment 
option for some students (rather than 
“outside their attendance areas,” which 
could refer either to school or district 
lines). Likewise, obtaining the number of 
magnet schools and their names is feasible, 
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even in large districts. Brochures 
describing these programs are routinely 
sent to parents. 

In SASS, the school was asked two distinct 
questions concerning special or magnet 
programs. First, it was asked “what type 
of school it is”: a “regular” school, or 
whether it was a school with a “special 
program emphasis” such as 
“science/math” or “performing arts” (or 
vocational/technical or alternative). The 
second response category should 
correspond to a “total” school magnet 
(without distinguishing between dedicated 
and partial attendance zones). The 
percentage estimate from the 1993 SASS is 
3 percent magnet using the first definition, 
which is higher by 1 . 1 percent than the 
AIR figure. However, the racial 
composition of the magnet schools in 
SASS parallels the AIR survey (57.9 
percent of the students are in schools 
greater than 50 percent minority in the 
former versus 56.5 percent in the AIR 
survey). Likewise, the SASS survey 
locates nearly 60 percent of the magnets in 
central city districts as expected. 

A second question in SASS asked whether 
the school offered a “magnet program. ” 
This could reference “programs-within- 
schools” in contrast to total school 
magnets, but for some schools a “no” 
answer to this question could filter out 
schools that did not have a curricular focus 
and should not be considered as magnets 
in the conventional sense. Assuming the 
latter possibility, the SASS estimate is now 
1.8 percent (1,466 magnet/special schools 
weighted 33 ). 

We now turn back and ask the degree of 
consistency between the district and school 



surveys. Certainly in some districts, it is 
likely that no magnet school would be 
selected when a district indicates it has 
magnet schools (maybe one in four schools 
are magnet schools in a typical district 
with more than 10,000 students), and only 
6 percent of the districts fit this pattern. If 
slightly more refined questions had been 
elicited from the district administrator 
(e.g., asking for the subset of magnet 
school names), some confusion would have 
been minimized. More problematic is the 
situation when a school identifies itself as 
a magnet, and the district indicates it has 
no magnet schools. (In some of these 
cases, several magnet schools were 
sampled, and the districts are known to 
have comprehensive “choice”plans. 34 ) 
Nonetheless, we were able to match 501 
districts where there was a “yes” response 
both to the magnet school question in the 
LEA survey and by one school 
administrator in that district (the weighted 
average number of magnets in large and 
mid-sized cities was 4.3 schools). Most 
districts did not, however, provide 
estimates as to the number of students in 
the magnet schools (They would need a list 
of the schools themselves to count 
participants. 35 ). 

This preliminary exercise demonstrates the 
potential for SASS to explore complex 
policy issues in certainly a more cost- 
efficient manner than large-scale 
retrospective surveys. AIR collected most 
of its information on magnet schools (in 
contrast to general desegregation 
information obtained in the initial 
interview) from a follow-up survey to the 
127 districts that had choice plans. 

Another phase of the AIR study involved 
districts that had received federal Magnet 
School Assistance Plans grants over 
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several years between 1985 and 1993. 

The grants cover a three-year time span, 
and 117 districts had obtained a least one 
grant. The survey gives some insight into 
the administrative infrastructure that 
implements these complex student 
assignment plans using choice on a 
district-wide basis. Many of the districts 
used their program funds to hire new 
teachers and staff development for the 
magnet schools (besides substantial 
investments at the district level in program 
specialists). Implementation of new choice 
programs also required large-scale 
outreach programs to attract new students 
and the development of more extensive 
transportation plans to handle district- wide 
choice. An interesting aspect of these 
choice mechanisms is the manner in which 
districts handled admissions, the priorities 
granted in -the admission process, and the 
maintenance of waiting lists. Obviously if 
there is high demand, individual 
applications have to be administered and 
centralized at the district office, even in 
medium-sized districts. The questionnaire 
offers some insight into the administrative 
processes that are amenable to descriptive 
“check-listing” as approximate summaries 
of administrative decision making. Other 
reforms based upon choice mechanisms 
(between-district plans, charter schools, 
and voucher proposals) need similar 
administrative structures to attract students 
from large geographic areas. 

District-Level Studies of School Reform 

In this section, the potential for linking 
SASS district-level data to the assessment 
of state-level reform efforts is examined. 
Obviously, a variety of optimistic and 
speculative assumptions permeate this 
evaluation of SASS’s potential as a 



baseline survey for ongoing comparative 
state-level studies. First, it is hoped that 
some program (and/or discipline specific) 
data can be successfully incorporated from 
CCD aggregate sources into SASS district- 
level surveys as noted above. Second, one 
has to assume that basic commonalities can 
be extracted from the large number of 
“state” systematic studies currently being 
conducted, even though they involve only 
a few high-profile states. 36 Third, one 
must believe that the methodological split 
between qualitative ethnographic case 
studies and larger scale semi-structured 
surveys will diminish. Fourth, there is an 
expectation that additional investigations of 
state-level differences using the 1993 
SASS can replicate certain findings from 
these reform studies. Finally, one has to 
assume that more comprehensive surveys 
on the effect of reform implementation 
will be administered at the state level to 
compare different types of policies. At a 
minimum, a more realistic understanding 
of the difficulties should emerge from a 
critical comparison of different methods, 
even in an area with clearly established 
standards such as mathematics. 

A valuable introduction to some of these 
issues can be garnered from a recent 
Michigan State report (Spillane et al., 
1995 37 ). The methodological framework 
for this study has developed from a series 
of evaluations (see Spillane, 1996, March) 
where the “key role” of LEAs in 
instructional policy making is apparent. 38 
Michigan’s reforms are representative of 
initiatives that are designed to radically 
restructure instructional practices in a state 
and incorporate a specific set of state- 
developed policy recommendations as 
outlined in “Essential Goals and 
Objectives” 39 that are linked to national 
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frameworks in mathematics (NCTM) and 
science (AAAS). The objective of the 
Michigan State study was to determine 
what local school districts were doing to 
reform mathematics and science education 
(i.e., what changes were occurring) and 
what “influences the way local schools 
make policy about mathematics and 
science education.” 

Nine districts were selected based upon 
geographic location, district size and urban 
type, social and ethnic composition of 
student population, and “reputation” for 
reform activity. It should be noted that 
the range of variation rather than statistical 
representation was the primary 
consideration in district selection. 
Accordingly, the study included two large 
mid-size city districts with high-minority 
population and high percentages of “free 
lunch” students, but it also included 
smaller low-minority rural districts with 
substantial percentage of students receiving 
“free lunch.” 40 The interview selection 
process within each district is more 
complicated. First, central office 
personnel with instructional responsibility 
were interviewed; second, for the two 
elementary, one middle, and one high 
school from each district in the study, the 
principal was interviewed; and third, 
teachers with the “lead role” in 
mathematics and science education were 
also part of the study. 41 The total 
number of interviews ranged from 13 to 
32 per district, and these open-ended 
interviews were then transcribed. 42 Six 
categories were used to code the first 
round of interviews: background 
information on the district, substantive 
ideas about mathematics and science, the 
efficacy of LEA policy, the opportunity 
for teachers to learn about policies, and 



local perspectives on state and federal 
policies. 

At this point, it is necessary to review the 
broad outline of national NCTM standards 
and some of their implications for state 
policies before district implementation can 
be discussed (this is classic top-down 
reform). At a general level, these 
standards outline a general set of topics 
organized around four basic themes 
(problem solving, communication, 
reasoning, and connections) and then more 
grade-specific recommendations for 
alignment of content coverage. 43 In this 
study, the state document Essential 
Principles was generally recognized as a 
set of new policies that required substantial 
changes in curricula, instructional 
practices, textbooks, and so forth, by all 
districts. The conclusion of Spillane et 
al., however, is clear: “The reform 
rhetoric masks significant variability across 
and within districts” (p. 34). While all 
districts indicated that they were 
implementing the new state guidelines, the 
details of specific reforms revealed distinct 
differences in the priorities given to 
different themes. Only three districts had 
moved beyond more routine topic 
identification toward substantive alignment 
as it relates primarily to two criteria 
(communication and reasoning) when 
compared to the other districts that gave 
more attention to other themes (problem 
solving and hands-on mathematics). More 
specifically, in these latter districts new 
concepts became new “labels” for old 
activities (“hands-on” became the same 
use of concrete materials — 

“manipulatives” and “integration of 
concepts” became more group activities). 
The rich discussion in this report suggests 
an underlying rank order in the 
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implementation of these reforms that are 
measurable when one asks about certain 
topics for certain grades. 

The findings for science demonstrate how 
seemingly parallel state frameworks 
generate qualitatively different levels of 
change. The AAAS science standards 44 
emphasized connections and common 
themes between disciplines, teaching a 
smaller number of central scientific ideas, 
and developing students’ ability to utilize 
scientific methods and technology. While 
all districts were either purchasing 
curriculum guides from outside sources or 
developing their own, these materials were 
only aligned topically to the state 
frameworks. In the four districts that had 
moved closer to state standards, the 
“boundaries” between conventional 
disciplines had been “softened,” but there 
also was more explicit attention to 
principles of “constructivist learning” and 
“conceptual understanding.” Description 
of these topics, as district administrators or 
lead teachers explained how material was 
presented differently, suggested an ability 
to actively translate these principles into 
the curriculum. In many of the other 
districts, reform was limited to “hands-on 
science” and these reforms, unfortunately, 
appeared to be quite similar to the old 
“cookbook laboratory experiments.” 
Likewise, integration of content from 
different disciplines (particularly 
mathematics) often evolved into team 
teaching, without adoption of a newer 
integrated curriculum. 

The Michigan State study then examined 
the more complex process of how these 
proscribed changes in classroom 
instruction successfully flowed downward 
from these new district policies. 45 



Without commenting on the specific 
findings at the school level, they found the 
“LEA actively engaged in instructional 
policymaking, both defining policy 
problems and crafting solutions to them” 
(Spillane et al. , 1995). This conclusion 
contrasts with conventional perspectives on 
local school districts as the passive 
“implementor” of state and federal policies 
(or more narrowly concerned with 
administrative and budgetary issues ) 
rather than directly concerned with 
instructional content. This active role in 
instructional policy appears to be a new 
development in these districts, and the 
variability between districts is more 
striking when specific organizational and 
historical factors are examined. It is 
important to note that formal or 
(traditional) “channels” of influence had 
definite limits. Neither curriculum guides, 
curricular materials, student assessment, 
nor professional developments were 
initially influential in shaping mathematics 
and science reforms when traditional 
methods were employed. In a narrow 
sense, most LEAs emphasized the simple 
coherence of topics and utilized lists 
(“what teachers should teach”) rather than 
the more radical restructuring of ideas 
about “substantive reform ideas.” There 
is no question in the Michigan State 
analysis that two state laws (a mandated 
core curriculum and fiscal penalties for 
poor performance) stimulated district 
administrators to “pay attention to 
instructional issues for the first time.” 
Despite this opportunity to use these 
mandates to leverage support for new 
reform agendas within each district, the 
distinct variation between districts in their 
responses, as summarized above, does not 
lend itself to simple a priori 
explanations. 46 
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The conceptual approach proposed by 
Spillane accounts for these differences and 
relies upon district organizational 
capacities, organized access to 
information, and the skill of individual 
administrators (such as knowledge and 
commitment). 47 A common theme across 
these resource capacities was the 
mobilization of individuals within a district 
into a more focused and organized 
collectivity. (“This interaction of 
organizational and individual resources is a 
key to understanding an LEA’s capacity 
for instructional reform. Spillane et al., 
1995). Before the statewide reform was 
initiated, most districts had limited 
structural capacities to initiate new 
instructional initiatives. Administrative 
structures were hierarchical and 
preoccupied with managerial and 
procedural concerns (i.e., simple 
mechanical compliance). 48 The 
development of resource capacity 
described in the Michigan State study have 
certain parallels to James Coleman’s 
(1990) analysis of organizational 
innovation and the creation of social 
capital in output-driven systems. In order 
to construct new curriculum material, 
district administrators first had to identify 
knowledgeable experts within the district 
(usually “lead” teachers) in each 
discipline, and then organize some 
sufficient numbers of these individuals 
willing to collaborate in a “new” 
enterprise. 49 

Second, links to external professional 
networks 50 provided access to discipline- 
specific knowledge for certain 
administrators and teachers. Besides 
providing opportunities to learn about 
these new reforms first hand from these 
professional organizations, they allowed 



participants to bring back to their districts 
a “sense of ownership of the reform 
agenda” which they then could 
communicate with more substantive 
conviction to other teachers. The third 
elements (time, funding, and labor) were 
resources that district administrators 
continued to control before and after the 
new reform initiatives. Small district size 
and state regulations often imposed serious 
constraints on the ability of districts to 
shuffle priorities in order to give more 
attention to new instructional issues. For 
example, staffing curriculum development 
committees and funding substitutes while 
regular teachers spent one week in 
professional development seminars 
required administrative skill (and not 
budgetary flexibility and available funds). 

These case studies provide strong 
evidence, in my opinion, that the local 
school districts will play a critical role in 
educational reform. State legislation has 
affected the broad parameters of reform in 
terms of proposing core curriculum, new 
statewide testing requirements, and funding 
incentives. But the state mandates cannot 
be directly translated into new instructional 
practices at the school level without a 
restructuring of the relationship between 
schools and district curriculum policies as 
demonstrated by these experiences in 
Michigan. At the same time, other reform 
strategies that have focused on individual 
school-level reforms (where the district is 
bypassed entirely) have not been a 
“stimulus to change in individual schools 
over time” (Elmore, Abelman, & 

Fuhrman, 1995). In the context of the 
Spillane arguments, individual schools do 
not have the administrative capacity or 
resources to mobilize curriculum reforms 
mandated by new standards (although a 
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limited number of high performance 
schools may have successfully 
implemented them prior to the 
establishment of these reforms 51 ). In 
Michigan, a permissive charter law will 
provide interesting comparisons of 
different strategies, even though these 
charters must be “sponsored” by school 
districts. 

The final question, of course, is the ability 
to consistently “track” different reforms in 
different states. 52 At least for 
mathematics and science, comparative 
“evaluations” have been started as part of 
the NSF state systemic program (SRI, 
1996), 53 and individual states receiving 
SSI grants have also conducted their own 
studies. 54 There may be sufficient 
information from all these studies to 
extract some common themes concerning 
the role of district administration in 
developing instructional content in their 
schools. Some precipitating event (usually 
new state requirements, tests, or 
curriculum policies) provides an 
identifiable context (when did this occur, 
what did the district think it would have to 
do, etc.). Then the process of 
implementation involves several key 
elements: How was the new curriculum 
content constructed? Who was involved, 
how long did it take, what financial 
resources were shifted? What was the 
impact of new testing standards? Were 
new forms of professional development 
organized? The articulation of different 
themes in the new mathematics standards 
suggests that how administrators “talk” 
about reform has some relation to what 
they have done to develop new material, 
and how they have gone about 
implementing “reforms” in their schools. 
And, these administrators usually have 



some “feedback” from principals, 
teachers, parents (and probably their 
school board and superintendent) 
concerning the “relative progress of 
change." 55 Michigan is not unique in this 
“movement” toward “standards based” 
reform. 56 The organizational structures 
of local districts have responded, maybe 
more out of necessity than principle, to 
these pressures for improved student 
performance from the state. District 
administrators have forged new 
connections with school staff and teachers 
to design a more coherent, but not always 
consistent, set of curriculum principles and 
instructional guidelines. In most cases, the 
districts have not passively removed 
themselves from the process and they have 
not allowed individual schools to mobilize 
existing capacities or develop new 
resources for these new standards. 



Conclusion 

The current and future utility of SASS is 
derivative of these relatively new state 
education reforms. 

The only comparative state data on the 
organizational capacities of districts and 
schools comes from SASS. Short-term 
student outcomes from state-level NAEP 
are important, but between-state variation 
often are not as critical as between-district 
comparison within a state to state-level 
policymakers. They are more concerned 
with their own performance systems and 
the quality of instructional capacities and 
resources within their own state (more 
importantly how they have changed over 
time). Accountability within existing 
governance structures is an active force 
driving these reforms. State education 
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commissioners are now more attuned to 
governors and state legislatures, and 
school superintendents are more responsive 
to their local constituencies. 

Administrative and school organizational 
processes have changed accordingly, and 
this change has implications for how a 
survey is organized and designed. 

First, district administrators are no longer 
exclusively concerned with routine 
budgetary matters. More important, the 
details (the dollars, the personnel counts, 
the number of students, closing old 
buildings, etc.) are now collected and 
reported on an accurate and regular basis 
in the CCD surveys. When the National 
Center for Education Statistics comes back 
for the same information from the same 
administrators, the “bureaucratic response” 
may vary. The numbers may have 
changed in the intervening months, certain 
types of details may never have been 
available, or the terminology may not be 
recognizable to administrators in certain 
types of districts. Large districts are 
fundamentally different from small 
districts. Accordingly, questions that 
overlap with CCD surveys should, in 
principle, be avoided or subject to a 
simple and quick review (at the end of the 
survey: just ask, “by the way, can you 
quickly confirm these numbers.” 57 ) 

If one asks district administrators questions 
that are more relevant to their day-to-day 
concerns and problems (and which have 
not been asked before), they can give more 
detailed, consistent, and informed 
responses. The questions themselves have 
to be logically simplified with more 
introductory explanations to establish 
common definitions. A major concern of 
districts is the increasing preference for 



“choice” mechanisms in student 
assignment (at a certain level, this is how 
public schools respond to market pressures 
for private schools, charters, and 
vouchers). The slow demise of the 
neighborhood school 58 presents new 
problems of matching parental preferences 
to more distinct educational offerings. If 
within-school reforms did not work, 
districts had to shift to district-wide 
magnet schools and more limited choice 
options. In the process, the allocation of 
instructional staff is also subject to similar 
pressures. Total-school magnet principals 
are usually allowed to choose most of their 
staff from any school in the district when 
the school first opens. What district 
administrators know (and what school 
principals do not) is the complexity of 
shifting students, staff, and finally federal 
and state money associated with programs 
for special populations between schools . 

In most cases, these processes are not 
random and cannot be reconstructed 
without understanding the process. 

At a minimum, district administrators can 
describe what they have done, or at least 
what they have been doing, since some 
“new” policies were implemented at some 
fixed point in the recent past. 59 The 
outline of these policies for student 
assignments has already been discussed. 

But the reform process also has to involve 
teachers and principals. Ironically, many 
teachers have strong ties and professional 
investments in the school where they are 
“employed,” but they are employed by the 
district rather than the school. Subject to 
a multitude of procedural constraints, 
teachers can be and are reassigned with 
only limited “choice” by district 
administrators. In the 1980s, arbitrarily 
shifting personnel between schools was 
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common. The Michigan State study 
provides a different perspective on the 
networks between teachers in different 
schools, administrators, and professional 
organizations that are emerging in 
response to new state content standards. 
The outlines of these organizational 
capacities and the resources (often more 
time than money now) that are necessary 
to mobilize before instructional change can 
occur are still controlled by local school 
districts. Despite strong pressures for 
decentralization and deregulation, schools 
themselves have demonstrated limited 
capacity to initiate reforms except in 
isolated cases. 

The question itself about the future status 
of a LEA survey, in many ways, mirrors 



the policy dialogue as discussed by 
Spillane et al. Maybe, if one continues to 
avoid the question, the significance of 
administrative structures will quietly 
disappear. This probably will not occur. 
States have chosen not to abolish local 
school districts, and only under extreme 
conditions such as receivership have states 
decided to administer districts with state 
personnel. The only alternative is to 
review, step-by-step, the implementation 
process of new reforms at the district level 
in each state. The administrative process 
reflects a common set of instructional 
themes and new accountability 
mechanisms. The major changes are 
compatible with a new and measurable 
discourse. 
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Appendix I 



STATE 


DISTRICT SAMPLING 
Dist School Ave. 


SCHOOL SAMPLING 
Dist School Ave. 


CCD 

Ave 


Dist 
> 10k 


Alabama 




42 


203 


4.8 


103 


221 


2.1 


10.0 


6 


Alaska 




15 


136 


9.1 


46 


196 


4.3 


8.7 


3 


Arizona 




50 


246 


4.9 


95 


170 


1.8 


5.0 


14 


Arkansas 




61 


173 


2.8 


126 


164 


1.3 


3.4 


2 


California 




99 


394 


4.0 


268 


416 


1.6 


7.2 


53 


Colorado 




31 


301 


9.7 


74 


173 


2.3 


7.6 


16 


Delaware 




22 


51 


2.3 


19 


72 


3.8 


8.0 




Dist of Col 




1 


35 


35.0 


1 


72 


72.0 


181.0 




Florida 




20 


400 


20.0 


55 


258 


4.7 


37.0 


19 


Georgia 




44 


286 


6.5 


97 


179 


1.8 


9.4 


17 


Hawaii 




1 


73 


73.0 


1 


94 


94 


238 


1 


Idaho 




36 


167 


4.6 


79 


166 


2.1 


2.3 


4 


Illinois 




94 


344 


3.7 


193 


283 


1.5 


4.2 


9 


Indiana 




61 


218 


3.6 


132 


184 


1.4 


6.2 


10 


Iowa 




72 


204 


2.8 


128 


169 


1.3 


3.6 


6 


Kansas 




57 


243 


4.3 


110 


161 


1.5 


4.9 


6 


Kentucky 




42 


201 


4.8 


98 


167 


1.7 


7.9 


3 


Louisiana 




29 


297 


10.2 


67 


225 


3.4 


20.5 


15 


Maine 




54 


173 


3.2 


105 


152 


1.4 


3.2 


0 


Maryland 




14 


293 


20.9 


23 


171 


7.4 


52.6 


12 


Massachusetts 




67 


212 


3.2 


157 


229 


1.5 


5.0 


7 


Michigan 




85 


329 


3.9 


189 


227 


1.2 


5.2 


16 


Minnesota 




72 


200 


2.8 


134 


171 


1.3 


3.7 


13 


Missouri 




76 


249 


3.3 


126 


178 


1.4 


4.1 


13 


Montana 




70 


137 


2.0 


155 


161 


1.0 


1.8 


1 


Nebraska 




71 


204 


2.9 


116 


170 


1.5 


2.0 


3 


Nevada 




11 


193 


17.5 


18 


119 


6.6 


21.3 


2 


New Hamphsire 




46 


135 


2.9 


76 


120 


1.6 


2.7 


2 


New Jersey 




93 


223 


2.4 


151 


194 


1.3 


3.9 


9 


New Mexico 




25 


188 


7.5 


62 


142 


2.3 


7.4 


7 


New York 




63 


384 


6.1 


201 


313 


1.6 


5.3 


7 


North Carolina 




41 


325 


7.9 


92 


184 


2.0 


14.9 


21 


North Dakota 




47 


136 


2.9 


130 


162 


1.2 


2.3 


2 


Ohio 




93 


239 


2.6 


155 


196 


1.3 


6.1 


10 


Oklahoma 




72 


230 


3.2 


235 


161 


.7 


3.2 


10 


Oregon 




53 


255 


4.8 


107 


170 


1.6 


4.1 


7 


Pennsylvania 




88 


225 


2.6 


159 


196 


1.2 


6.0 


6 


Rhode Island 




23 


158 


6.9 


35 


106 


3.0 


8.5 


2 


South Carolina 




38 


257 


6.8 


70 


164 


2.3 


11.6 


13 


South Dakota 




48 


131 


2.7 


112 


164 


1.5 


3.5 


2 


Tennessee 




35 


258 


7.4 


86 


189 


2.2 


10.9 


12 


Texas 




84 


442 


5.3 


291 


413 


1.4 


5.9 


45 


Utah 




20 


257 


12.9 


31 


170 


5.5 


17.8 


11 


Vermont 




62 


91 


1.5 


92 


108 


1.2 


1.3 


0 


Virginia 




35 


275 


7.9 


92 


188 


2.0 


13.4 


15 


Washington 




56 


298 


5.3 


117 


197 


1.7 


6.8 


18 


West Virginia 




28 


216 


7.7 


55 


178 


3.2 


16.6 


6 


Wisconsin 




64 


219 


3.4 


126 


170 


1.3 


4.8 


7 


Wyoming 




55 


96 


1.7 


50 


131 


2.6 


. 


0 


Total 




2573 


11379 


4.4 


6459 


9333 


1.7 


4.4 
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The formal governance of public schools itself is another topic there is little agreement as to what the system 
actually looks like.” (The Twentieth Century Fund, 1992) 



For example, Baker (1996) and Stodolsky (1996) implicitly link individual teachers to classroom instruction 
practices through a greater attention to content-specific disciplines. 

This paper unfortunately can not address any of the important organizational parallels or differences between 
sectors (Baker, Han, & Keil, forthcoming). 

How this centralized and integrated political culture survived reform-oriented competition in the 1980’s is another 
question. 

This argument follows the discussion presented by Scott and Meyer (1987). 

The percentage of funding from states sources (45.6 percent) now exceeds local sources (44.7 percent), State 
Comparisons of Education Statistics: 1969-70 to 1993-94. 

The historical sources of this situation are traditional localism and federalism, mistrust of government, and 
political design. 

The original argument was presented by Meyer, Scott, and Strang (1987), but has not been updated. Increasing 
sophistication in aggregate district-level and school-level data in CCD (supplemented by individual data from SASS 
school and teacher surveys) provide untapped resources to verity changes in these theories of bureaucratic 
complexity. 

Carnegie Forum on Education and the Economy, A Nation Prepared : Teachers for the 21st Century , the Report of 
the Task Force on Teaching as a Profession (1986). The most detailed review is E. Boe and D. Gilford, Teacher 
Supply , Demand and Quality (1992). 

The most comprehensive review of SASS-related research is contained in Ingersol (1995). 

The school-level question asked only if there were any unfilled position, not how many. It should not be expected 
therefore that aggregate school-level numbers will provide precise estimates. 

The limitations of the LEA survey, which can only ascertain how many teachers were new hires in t hat district , 
are quite apparent compared to individual data. A substitute measure based on net aggregate change from CCD 
would probably be satisfactory. 

The sophisticated weighting of schools and districts ensures close estimates. Nonetheless, there are some 
inconsistencies in district-level responses such as counting K-12 but including pre-kindergarten counts in the CCD. 
The time frame sometimes is important with districts giving numbers as of the date when they are filling out the 
questionnaire rather the date specified in the SASS survey form. 

For example, the 1993-94 SASS public school sampling design is based upon teacher counts from the 1991-92 
CCD (see Kaufman et a!., 1996). 

When a LEA does not respond, simple variables, such as district size, for example, are treated as missing in SASS 
user files. 

There are seven functional categories including special regional purposes. Approximately 1,197 districts (1992-93 
CCD) have either no (or only one) school, and no students (but some also report FTE teachers). Schools in these 
districts are excluded from the sampling frame and therefore these LEAs properly are not included in the district 
frame. In the 1993-94, there was a process to sample these teachers, but only a small percentage were actually 
found to be teaching in regular districts. It is tempting to recommend a “footnote and exclude” philosophy for 
these districts with minimal staff and small numbers of students. 
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17. 

18. 



Based upon CCD estimates, only 53.8 percent of the total public school FTE are teachers. 

Individual decisions are often made by the principal or hiring committee from filtered lists prepared by central 
office staff. 



19. 



20 . 



21 . 



22 . 



With a five-year interval, the district and school survey should ask whether policy changes occurred during this 
period and if so in what year. 

In many large districts, attendance zones can be complex (particularly when choice options are available to some 
students), and they are established (and often adjusted on a yearly basis) exclusively at the district level'. 

The number and scope of these programs is well beyond the scope of this paper, but evaluation of these programs 
has frequently involved representative samples and structured surveys. 

The discretionary factors that districts can control were primarily measured by teacher characteristics 
(undergraduate major, highest degree, type of certification, etc.) rather than district-level data such as base 
salaries, etc. 



23. It should be noted that “climate” variables constructed from aggregating individual teacher responses to the school 
level had large impacts. While also accounting for some of the missing schools, aggregation is an alternative 
strategy to estimate higher-level units. 

24. These equations allow one to estimate whether opposite race teachers (minorities teaching in white schools and 
whites teaching in minority schools) had higher salaries controlling for the racial composition of the district. The 
major effect was for Hispanic male teachers to have higher salaries than white male teachers, and this effect was 
not dependent upon the percentage Hispanic in the school or the district. 

25. In some preliminary estimates, the average number of schools per district for the state of Michigan would increase 
from 1.2 per district to 4.8. 



26. Florida has a total of 67 districts. 

27. There are still over 6,000 districts with less than 600 students. Under this proposal fewer schools would be 
sampled and a brief (if any) LEA instrument could be administered, thereby reducing total burden. 

28. In North Carolina, there are seven schools with student populations greater than 25,000 and an average of 19.7 
schools (the range was 10- 37 schools) were sampled. Among the 14 smaller districts (population between 10 and 
25 thousand), an average of 6.8 schools per district (the range was 3 to 13 schools) were sampled. 

29. The size of the district when a desegregation plan was first implemented (not a district’s size when its current plan 
is assessed) is important to consider in order to avoid confounding growth in student population due to racial 
composition of the district and its metropolitan context with the long term effects of the plan independent of its 
specific components. 

30. Complex student assignment plans utilizing choice are typically not employed in smaller districts. In their initial 
plans, all-black schools were closed and attendance boundaries or grade structures were changed. 

31. There was an additional condition concerning special needs students. 

32. For rough comparisons, the AIR study estimated that only 230 districts had magnets. This was a telephone survey 
that included lengthy descriptions of what magnets were and their objectives for desegregation purposes. 

33. The published figure in 1993 Statistical Profile is 6.5 percent in reference to programs offered within schools in a 
manner comparable to bilingual or Chapter One (Table 2.4). 
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The number of districts with this erroneous classification was only nine after the revised definition of a magnet 
school was used and some may have been nonresponses, such as Chicago and St. Paul. 

The “open enrollment” question did not provide meaningful numbers. In most small districts (with one type of 
school for each grade level), all students were in “open enrollment” programs. Between school transfers is an 
unfamiliar concept. 

The criteria for state selection are driven by standards-setting criteria, primarily increases in the number of 
mathematics and science credits required to graduate from high school. Comparable state data on credits, 
revisions of guidelines to align with NCTM standards, and test requirements for graduation are currently reported 
in Table 17 in State Comparisons. 



This ongoing study was funded by the Michigan Statewide Systemic Initiative under the direction of James P. 
Spillane. 

This an ongoing three-part study that is properly classified as a “policy implementation” research. 

The development of these new standards is contained in Thompson, Spillane, and Cohen, The State Policy System 
Affecting Science and Mathematics Education in Michigan (1994). 

This type of district selection strategy, common in most state-level evaluations, implicitly reflects the interest to 
find the widest range of different types when only a limited number of districts can be studied in depth. 

In principle, 36 schools would constitute the selected school sample for these nine districts, although in the smaller 
districts fewer schools were part of the sample. The 1993-94 SASS sampled 227 schools (7 percent) in Michigan, 
based upon the number of teachers in the state. In general, Michigan (558 districts) has a large number of smaller 
districts with 27.1 percent more of its students in districts less than 2,500 compared to national levels. 

More structured interview protocols for district-level personnel were developed in the Reform Up Close study. 
There are certain characteristics of district-specific policy that appear to be ’’common”: is there a “framework 
document that the district has adopted, who decides what textbooks will be used, is there a testing program in the 
district, and have there been changes in graduation requirements. 

This approach differs from the more detailed taxonomy used in the Reform Up Close study (see M. Leighton and 
J. Mullens, Measuring Curriculum Content: The Status of Recent Work) designed to measure instructional content 
at the classroom level. 

The National Research Council has proposed a different set of standards. 

The actual classroom implementation of these reforms is the third phase of this study. Of course, how 
comprehensive district-level curricula facilitates more fundamental changes in instructional practices within schools 
(even when background variation in content knowledge is considered) provides a rigorous test for this district 
mediating theory. 

From the perspective of developing a district-level survey in the context of SASS, it is not clear that retrospective 
accounts at a single point in time can reconstruct how more successful and articulate themes were developed by 
administrative personnel. 

Six factors are identified in the study: knowledge, commitment and disposition, time, funding and labor, 
professional networks, trust, and collaboration. 

The obstacles presented by bureaucratic layering (particularly in large districts) have to be considered a factor in 
the slow implementation of reform. It seems that districts had the necessary resource capacities, but had to make 
specific decisions to mobilize content knowledge and utilize this expertise in a different manner. Different 
allocations of personnel, time, and funds were critical in the “crafting” of district-wide policy. 



From the descriptions in the study, the necessary “critical mass” was a mixture of self-selection (teachers 
volunteering themselves to lead the reform) and decisions by district administrators about their level of expertise 
Apparently, conventional selection standards (no rules seem to have guided the selection) for committee 

participation were not followed and a decision was made to include teachers from most or all schools to bolster the 
argument for representativeness.” 



These include NCTM, Michigan Council of Teachers of Mathematics, Michigan Partnership for New Education’s 
Frameworks Project, Michigan-funded Mathematics and Science Centers. A substantial amount of federal 
Eisenhower funds flowed to the organizations through professional development programs (allocated to the 
districts). 



The issues in school-level reform, primarily deregulation to promote autonomy and innovation, often find 
themselves limited to “relatively successful schools, to which eligibility was generally limited, did not find much 
need to embark on wholesale change, and used deregulation as one of many resources to support innovation ” 

(Elmore, Abelman, & Fuhrman, 1995). 

Unfortunately, the more detailed case studies have focused on implementation in individual states and the more 
comprehensive, comparative state analyses focus on national standards without providing analytic methods to 
examine between district variation within states. 

CPRE s new Center, funded through the Governance Institute, has proposed additional analysis of state and local 
reform policy in these 26 states. 

It is extremely doubtful that many of the state-level studies are comparable in insight and depth to the Michigan 
State study. 

The Michigan State study included some interesting material on “complaints” (i.e. , why the state policies were 
unreasonable,” why they didn’t have money or personnel to prepare the mandated guides, etc.) that help 
distinguish the relative progress of different districts. 

In a survey of 50 state school superintendents or commissioners, 43 claimed that they were revising their 
assessment and accountability systems in accordance with these principles (Elmore, Abelman, & Fuhrman, 1995). 

Most of the first 22 questions are eliminated by this criteria. 

Fixed geographical attendance boundaries were more amenable to sophisticated geocoding schemas and 
computerized transportation programs. These administrators didn’t have to visit a school or talk to a parent. 

Despite the idiosyncratic labels embedded in district-wide reforms, there are common features below the surface. 
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USE OF EDUCATION INFORMATION SYSTEMS 
WITH THE SCHOOLS AND STAFFING SURVEY: 
HOW CAN SASS BE LINKED TO SCHOOLS? 

Rolf K. Blank, Council of Chief State School Officers 



The Schools and Staffing Survey (SASS) 
has proven to be a very important and 
useful multi-purpose survey of teachers, 
administrators, and school systems. The 
next iteration of the SASS now being 
planned for 1998-99 school year can build 
upon a record of success and 
accomplishment from the 1988, 1991, and 
1994 surveys. SASS is accomplishing 
some important goals; in the 1999 survey 
it can reach for a new goal of relating 
survey data on teachers to more detailed 
data from school information systems on 
the students teachers teach and the schools 
where they work. 

SASS results provide important national- 
and state-level statistics for American 
educators about critical areas such as 
characteristics of the teacher and 
administrator force in elementary and 
secondary schools, basic descriptive 
statistics on dimensions of schools, and 
reliable data on the preparation and 
experience of teachers in their assigned 
fields and positions. SASS provides 
critical national statistics on projections of 
teacher supply and demand and key 
variables for analyzing trends in teacher 
turnover and hiring. 

I approach this paper for the National 
Center on Education Statistics (NCES) 
from the perspective of educational uses of 
SASS and the data produced from the 
surveys. SASS meets several important 
purposes: reliable, periodic statistics for 



monitoring and reporting on the status of 
education systems to Congress, federal 
agencies, and states; and a valuable source 
of data on teachers and administrators that 
is useful for analyzing current conditions 
in K-12 education and projecting needs for 
educators. 

My main position in this paper is that the 
current design of SASS could be 
significantly strengthened through linking 
with education information systems and 
that it would provide important data on the 
characteristics of American schools and 
how education is carried out within them. 
My suggestions and proposals for use of 
data from education information systems 
are based on my assessment of the 
strengths and weaknesses of the current 
survey and my views of the needs of 
education decision makers and educators 
for better information about schools and 
staff. 

State and Local Education 
Information Systems Versus SASS 

Strengths of SASS Design 

SASS is based on surveys of individuals 
from stratified national and state samples 
of schools. The strength of the design is 
in providing reliable national estimates 
from samples of teachers and 
administrators with sufficient numbers of 
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respondents and items to provide both 
detailed descriptive information and 
relational analyses using a large number of 
variables. Another strength of the design 
is providing reliable state-level estimates 
across a wide range of variables describing 
teachers, administrators, and schools. In 
some areas of statistics about education, 
SASS provides breadth and depth of 
information not available in any other 
national or state survey, including: 

• Teacher and administrator 
background 

• Teacher education and experience 

• Current assignments 

• Pay and benefits of educators 

• National projections of supply and 

demand 

• Student enrollments and staff 
characteristics analyzed by key 
demographics-size, type of 
community, state 

SASS is designed from a research-based 
model of key statistics for understanding 
and explaining the operation of the 
teaching force in schools. Using a sample 
survey, SASS can ask more questions from 
a sample of teachers and schools than 
would be possible or feasible in a universe 
or population data collection. A sample 
survey across states also provides a 
standardized method of collecting data on 
basic elements of the education system, 
such as number of schools, students, 
special programs within schools. States 
collect data on these variables but they are 
hard to standardize. 

Limitations of SASS Design 

The SASS design emphasizes certain 
purposes and uses. The main limitations 



of SASS as a general-purpose education 
survey are: 

• Information about the students in 
schools 

• Lack of curriculum and instruction 

data 

• Student outcomes 

• How schools are organized and 
how resources are used 

• Capacity to analyze equity of staff 

and resources allocation in relation 
to student characteristics 

SASS has further limitations at the state 
level. Due to limitations of sample size in 
each state (100-200 schools), important 
uses of SASS data at the national level are 
not possible at the state level, such as 
analyzing extent of teacher turnover by 
field, projections of teacher 
supply /demand, and trends in preparation 
of teachers by field. 

Design of Education Information Systems 

State, local, and school education 
information systems are typically based on 
universe data collection. These data 
systems have been designed mainly from 
the requirements of state legislation and 
district policy-that is, the systems are 
designed to provide counts and tabulations 
on students, schools, and educators in 
order to satisfy laws and policies 
concerning education accountability, 
monitoring, and reporting. An example 
would be reporting the extent to which 
schools have state-certified teachers, or 
assessing the extent of improvement in 
rates of graduation. 

Recently, more states and districts are 
designing data systems based on rational 



planning of the multiple purposes and uses 
of data at all levels, and the efforts include 
systems for linking and transferring from 
classroom to school level to district to 
state. However, even the most recently 
developed “unit record systems” such as in 
Texas, Florida, and Ohio have not 
attempted to expand the breadth of the data 
into many of the areas covered by SASS, 
such as teacher background and education, 
professional development, attitudes, or 
conditions in schools (CCSSO 
[unpublished analysis of state 
accountability and indicator reports], 

1996). 

All states have an education information 
system that involves collecting data from 
all districts and schools. About 45 states 
maintain annual databases for all teachers 
and administrators with their 
assignments/courses and other information. 
About ten state systems include student 
records with student background data and 
annual data on variables such as student 
activities, courses, and test scores. 

Why Use Education Information 
Systems? 

Given the main differences between the 
purposes and scope of data in SASS and 
state or local systems, what are major 
reasons for trying to link data from 
education information systems to the 
SASS? What advantages and benefits are 
there are for NCES, the sponsoring 
agency? What advantages and benefits are 
there for states and for local districts and 
schools— that is, the expected partners in 
the project? What are the advantages and 
benefits for customers and users of SASS 
and education data? 



There are three possible approaches to 
linking SASS with data from education 
information systems. All of these 
possibilities reflect the goal of increasing 
data analysis and reporting at the state-by- 
state level. SASS has made strides in 
providing state-by-state data on education 
(NCES, SASS by State, 1994). My view 
is that SASS can significantly strengthen 
the useability and relevancy of the Survey 
to educators by linking to education 
information systems that will greatly 
augment the capacity for reporting at the 
state level and developing within-state 
analyses. 

State Aggreeate Statistics 

SASS results are used to produce 
indicators at the state level. The National 
Education Goals Panel, Council of Chief 
State School Officers, and the National 
Science Foundation are among users of 
state-level statistics, mainly data about 
teachers, which are generated from SASS. 
It would be possible to generate state totals 
and averages for indicators not available in 
SASS, such as: 

• Course enrollments 

• Enrollments in special programs 

• Graduation rates 

• Student and teacher transfers 
between schools and districts 

• Detailed data on teachers by 
assignment 

These state aggregate statistics could be 
generated by state education information 
systems using definitions and categories 
supplied by SASS, allowing a match to 
SASS statistics. For example, state data 
could be used to report the percentage of 
teachers in each state in the subjects/fields 
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defined on SASS questionnaire that are 
over a given age or are male vs. female. 

In 1991, NCES conducted a field trial with 
ten states by asking them to report state- 
collected data on specific SASS items, 
including teacher assignments, 
certification, and special program 
enrollments. A workshop held by CCSSO 
to analyze the results showed that there 
were differences in how states defined and 
reported some of the variables in 
comparison to SASS and in comparison to 
each other. State data managers did see 
the possibility of reporting data requested 
by NCES at the state level that are not 
reported by SASS, or variables that do not 
have sufficient detail at the state level. 

State managers saw strong benefits for 
decreasing burden on schools of reporting 
available data from information systems to 
avoid duplication or further need for 
expansion of SASS. 

How could this approach (ttl) be carried out? 
NCES would need to contract with each 
state, or an organization that would 
subcontract with each state, to produce 
data according to specifications and 
definitions defined by SASS. The amount 
of money would not be large for each 
state, but a contract would provide for a 
timetable, staff to perform the work, 
reporting schedule, a state commitment, 
and assurance from NCES about uses and 
reporting of the data. The advantage of 
using state aggregate statistics reported by 
states is that SASS could obtain additional 
state-level indicators in key areas of policy 
and educator interest, such as progress of 
secondary students to higher level courses, 
with relatively little additional cost and 
with the addition of 50 data collection sites 
(state departments of education). The 



disadvantage of this approach is that data 
would not be available for analysis down 
to the district and school level. 

State aggregate data on science and 
mathematics enrollments, teacher 
assignments and demographics, and 
teacher certification have been collected by 
CCSSO from SEAs since 1990. 

Currently, 35 states report on all of the 
state aggregate statistics. Forty-five states 
report on teacher assignments for grades 7- 
8, 9-12 (Blank & Gruebel, 1995). 

CCSSO also collects state policy 
information from all states every two 
years. The policy information on 
graduation requirements, standards, 
teacher certification, attendance, 
accreditation, and policies on time can be 
made available to SASS (CCSSO, 1995). 

States Report Data at State and School 
Level 

SASS selects a sample of schools for the 
nation and for each state. Teachers and 
administrators are surveyed from the 
sample of schools. At the school level, 
the SASS survey collects data on student 
and teacher characteristics such as 
race/ethnicity and on enrollments in 
programs such as Title I and vocational 
education. In 1994, SASS requested data 
on a sample of students from each school 
in order to determine school-to-school 
differences among students. 

SASS is limited in reporting on and 
analyzing differences among states in 
school-to-school variation in areas such as 
student background, teacher 
characteristics, and school resources and 
materials. Four important questions could 
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be answered by linking SASS with state 
information systems for a sample of 
schools in each state. 

(A) Is there equity of access, by state, 
to well-prepared teachers and 
resources for excellent teaching for 
students from schools with varying 
composition and location? 

These data would provide the kind of 
research analysis that Jeannie Oakes, Iris 
Weiss, and others have conducted with the 
National Survey of Science and 
Mathematics Education (see, for example, 
Weiss, 1994). The NSSME surveys 
(1977, 1987, and 1993) were conducted 
with a national sample of teachers and 
schools. The results of the analyses 
showed significant variation in the 
allocation of well-prepared teachers to high 
minority vs. low minority and high-SES 
vs. low-SES schools. Also the issue of 
course taking differences by school/ student 
characteristics could be studied. This kind 
of analysis can now be carried out with 
SASS data at the national level. 

(B) What is the rate of teacher turnover 
and return to teaching, by state, 
across different subjects/fields? 

Little can be done with teacher 
supply/demand questions at the state level 
using current SASS samples. By adding 
universe data from all schools in SASS 
sample, including teacher assignments, 
demographics, number of years teaching, 
etc., NCES could join with states in 
analyzing turnover by field/ grade level and 
by school characteristics as well as 
analyzing some data on teachers returning 
to teaching and new teachers. 



(C) What are differences in curriculum 
offerings, programs, and course 
enrollments according to 
differences among schools and 
students? 

Many states, not all, would be able to 
report school-level data on courses and 
programs. Other states would have to 
request the data from districts or schools. 
These data would not provide data from 
teachers on subject content and instruction 
(see below) but they would be able to 
provide universe data for all students in a 
school to determine differences by school 
characteristics matched with teachers in the 
same schools. 

(D) What are the differences in 
expenditures by function (classroom 
teachers, administrators, benefits, 
buildings, textbooks, materials, 
technology) across school 
differences? 

Recently, systems have been developed for 
coding and analyzing education 
expenditures down to the school level. 
These systems involve a methodology for 
coding all education costs and allocating 
costs on a per school and per classroom 
basis. SASS produces important and 
useful data on teacher and administrator 
salaries and associated costs. If a state, 
districts, and schools used the same 
accounting system, SASS data could be 
linked to data at school levels and costs 
other than salaries could be analyzed to 
determine how education funds are used 
across states and by school characteristics. 

How could this approach (M2) be carried out? 
NCES could contract with states to report 
universe data on all students, teachers, 



courses, programs, and costs for schools 
selected into the SASS sample. This is 
based on the assumption that the SASS 
school sample is representative of schools 
in the state. If some additional issues 
pertaining to school organization, 
programs, and curriculum were desired, 
such as effectiveness of the middle school 
approach, additional schools would need to 
be selected. 

Data on the universe of teachers and 
students in the sample schools would allow 
for analysis of the questions about equity, 
teacher turnover, curriculum, and finance. 
NCES could establish a system of 
technical assistance to states for sampling 
and for matching data definitions in SASS 
with definitions used by states, or for data 
collection and reporting. This model of 
assistance to states is employed by Centers 
for Disease Control in its Youth Risk 
Behavior Survey that is conducted in 
collaboration with states. 

School as Unit of Analysis— Working 
Directly with Schools and Districts 

Data from schools and districts. A third 
approach would be to expand the number 
of variables for school-level data collection 
through education information systems. 

The 1994 SASS tried a student record data 
collection using a sample of students in 
selected schools. This was designed to 
provide statistics on the characteristics of 
students in each school. SASS could also 
link to school-level data systems and 
expand the kinds of questions to be asked 
at the school level. This model assumes 
that SASS could send a data diskette 
directly to a school and they could transfer 
school-level data directly to the diskette. 
NCES may consider a small payment to 



each school to assist with the data 
reporting/transfer. 

By drawing on existing school records and 
data systems, SASS could ask a focused 
set of questions that would get at the 
following key areas of data that would 
reflect key policies and school decisions, 
operations, student and teacher 
assignments, and student activities and 
progress. The kinds of data envisioned 
with this approach are similar to the kinds 
of items in the NELS surveys. All of the 
variables identified in #2 could be 
collected with this approach. Additional 
variables that could be accessed: 

• Changing student and teacher 
composition: number of new 
students per grade per year, 
number of new teachers per year 
per subject/grade 

• Allocation of materials, texts, and 
resources: e.g., data on the types 
of books, equipment, and other 
curriculum materials and how they 
are allocated 

• Student outcomes: type of student 
assessments and other methods of 
evaluating progress, how 
assessment data are used (public 
reports vs. teacher diagnostic and 
curriculum planning), aggregate 
school or class-level scores on 
tests, student retention and 
promotion, suspensions and 
transfers/reassignments 

• Curriculum information: data that 
reflect curriculum and instruction in 
schools may be obtained from 
information systems, including 
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course offerings, levels of courses 
and enrollments, teachers assigned 
to courses 

o Data on teacher inservice and 

professional development: schools 
or districts would be more likely to 
have data on professional 
development in an information 
system, possibly as part of their 
personnel files. To the extent these 
data are annually updated, a wealth 
of data on current professional 
development activities may be 
available 

How could this approach (ft 3 ) be carried out ? 
This school or district approach would add 
a major data collection component to 
SASS. Several thousand schools in the 
SASS sample would receive an additional 
request. By 1999, data systems and access 
to personal computers in districts and 
schools would be sufficiently developed to 
allow a computerized data collection form, 
saving time and effort for school 
personnel. Data systems and coding 
structures may be common enough to 
allow a direct data entry/transfer approach 
that would expand the amount of 
information that is currently collected from 
schools. The expansion of data collection 
from school level may not be possible if 
SASS relies on paper forms to be filled out 
at the school level. But, even if this 
method must be retained, a number of 
additions could be made at the school 
level. 

Schools are a key unit of analysis for 
research on education reform and analyses 
of education programs and curriculum, use 
of resources and staff within schools, and 
efforts to improve the skills of staff. 



Efforts should be made to collect and 
provide much broader access to data about 
schools and what goes on in schools, at 
both national and state levels. 

Incentives for Schools, Districts, 
States 

NCES and SASS needs to strongly 
consider an approach to data collection that 
will provide some incentives for 
cooperation. NCES and federal data 
collection has typically approached 
education units as “data providers.” Users 
are viewed as Congress, researchers, or 
policymakers. Educators know this. 

Surveys are viewed as a burden, 
particularly those for which they seldom 
see results or direct feedback. In order to 
make use of Education Information 
Systems, NCES/SASS will need to 
carefully think through and plan 
incentives. Here are suggestions: 

® Money: Small amounts go a long 
way toward reducing negative 
responses 

® Direct feedback: Give back school 
averages, state averages, or 
national averages as soon as they 
are available to all respondents. 

Give them back a sample of the 
data results that are allowable 
within the law. Make it a two- to 
five-page feedback of some kind 

® Give educators something else in 
return: NAEP has alleviated some 
hard feelings by allowing schools to 
keep calculators, science 
equipment, and other assessment 
materials. A report on the last 
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SASS survey could be given to 
each school when the requests are 
made 

Raise the visibility and importance 
for educators: A letter from a 
congressman, senator or state 
superintendent could be enclosed 
saying how the survey and data are 
important 

Invite educators to a workshop in 
each region or state: NCES puts 
lots of money into the Cooperative 
Statistics program with states. Use 
some of it to give and get feedback 
directly from educators in states 
about data and their uses 

Think creatively about education 
data helping education: Data are 
often viewed as part of monitoring, 
accountability-an external 
requirement. How can it be 
viewed and provided as a support 
to educators? Efforts to build 
bigger and better databases will not 
be widely used or appreciated if 
this question is not addressed 
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COLLECTING REPRESENTATIVE DATA ON SCHOOL RESOURCES: 
UNDERSTANDING THE LINKAGE BETWEEN ADEQUACY, EQUITY, 
AND OPPORTUNITY TO LEARN THROUGH SASS 

Jay G. Chambers, American Institutes for Research 



ImtrodlMCtitm and Background 

The purpose of this paper is to suggest 
ways in which the 1998-99 Schools and 
Staffing Survey (SASS) might be 
redesigned to examine patterns of resource 
allocation in public and private schools in 
the United States. The paper assumes that 
the reader has a basic knowledge of the 
current structure and content of SASS. 

This paper argues for taking advantage of 
the opportunity that SASS currently offers 
to expand and improve upon the resource 
data at the school and district level. It 
suggests some significant reconfiguration 
of the ways in which personnel data are 
gathered at the school and program level. 

It argues that SASS could serve as the 
foundation for considerable work of 
interest to other agencies within the U.S. 
Education Department and could be 
utilized as a centerpiece of data analysis 
for studies of educational programs. 
Moreover, an enhanced version of SASS 
could reduce the need for more detailed 
fiscal data collection at the school or 
program level. SASS also could enhance 
the student-level data collection to improve 
the quality of information on how 
individual students are being served, and 
SASS could begin to consider ways of 
linking to other databases on student 
outcomes to address productivity issues in 
education. 



The paper begins with a definition of what 
is meant by the term school resources and 
the importance of these data. This is 
followed by a discussion of an alternative 
source of data on school resources: 
namely, the fiscal reports of the Common 
Core of Data (CCD). The major portion 
of this paper is devoted to ways of 
redesigning SASS for the purpose of 
gathering better data on school resources. 
This section discusses ways of gathering 
data on the quantities of school and district 
personnel, the qualities of school and 
district personnel and compensation, 
information on the state regulatory and 
funding environment within which schools 
operate, utilizing the existing student 
questionnaire to gather data on service and 
cost profiles of students, the importance of 
linking SASS to other databases, and, 
finally, what we might give up in SASS to 
make room for the proposed changes and 
revisions. 

What Is Meant by School Resources? 

For the purpose of this paper, the word 
resource can be used interchangeably with 
input. School resources are the ingredients 
or inputs used in the educational process in 
the nation’s public and private schools. 
Among these ingredients are the 
characteristics and capabilities of the 
students themselves as well as the school 
personnel and non-personnel resources 
used at the school and district level to 
organize and provide educational services. 
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Each student arrives at a school with a 
unique set of endowments that when 
combined with the resources provided by 
the school produces a certain collection of 
outcomes, i.e., student performance, 
behaviors, and attitudes. 

Among the school resources are the 
variety of personnel who come into direct 
contact with students. These personnel are 
characterized by a set of attributes, 
abilities, and skills. Their job titles 
describe the ways by which they interact 
with students, while their personal 
characteristics provide information that 
may relate to their potential for generating 
outcomes for students. While there are a 
number of other important non-personnel 
resources (e.g., computers, televisions, 
and other technology items as well as 
access to the Internet and its utilization) 
which affect the process, this paper will 
focus attention on the personnel quantities 
and characteristics. The fact that more 
than 80 percent of public school district 
budgets are devoted to personnel costs 
suggests the importance of understanding 
how this input is allocated and utilized 
among different kinds of schools. 

Why Are Data on School Resources 
Important? 

By understanding the patterns of utilization 
and allocation of personnel resources, 
policymakers can increase their ability to 
address issues of equity, adequacy, and 
opportunity to learn. At the heart of the 
issue of adequacy are measures of access 
of the nation’s youth to educational 
resources, while the critical equity issue is 
the distribution of those resources among 
different types of communities and among 
students with different educational needs. 



For all children to have equal opportunity 
to learn, it is a necessary condition that 
they have similar access to an appropriate 
level of resources. The next step in this 
process is linking school resources to 
student outcomes. Increasing our 
understanding of the linkage between 
resources to outcomes is going to be 
critical if the push for higher academic 
standards as a cornerstone of school 
reform coming out of the recent Education 
Summit is to have any meaning. 

The value of school resource data may be 
found in the kinds of studies that could be 
done. 

Resource allocation studies. What kinds of 
resources are being allocated to different 
programs? What are the quantities and 
characteristics of the resources being 
utilized in different communities? How 
are services being delivered in the nation’s 
schools? To what extent are different 
kinds of services being offered and how 
are resources organized for the provision 
of services (e.g., through self-contained 
classrooms, integrated or inclusionary 
programs, pull out programs, 
departmentalized instruction)? What other 
kinds of supplemental or related services 
(e.g., physical therapy, occupational 
therapy, speech therapy, counseling, social 
services) are being offered? What are the 
patterns of inequity in resource allocation? 
What district characteristics contribute to 
differences in resource levels? To what 
extent do community factors play a role in 
resource allocation? How much can be 
affected by policy changes? 

Estimates of programmatic costs. What are 
the patterns of variation in the allocations 
of resources among different educational 
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programs? How well do these programs 
relate to the patterns of variation in student 
needs? What are the implications for the 
adequacy and equity of existing patterns of 
resource allocation? 

Educational productivity. What are the 
relationships between school inputs, 
student characteristics, and outcomes? 
Which kinds of school resources make the 
most difference? Are there differences 
among types of communities in the 
effectiveness of school resources? To 
what extent can differences in outcomes be 
accounted for by differences in school 
resources? 

School finance reform. What impact do 
different school finance formulas have on 
the levels and allocation of educational 
resources among different communities? 

To what extent do differences in district 
versus school characteristics influence the 
level of resources allocated to individual 
schools? 

School labor markets . What are the 
differences in the costs of different school 
personnel? What factors affect the ability 
of different local communities to recruit 
and attract qualified personnel? What 
kinds and combinations of school 
personnel are being utilized in different 
communities across the nation? How do 
such personal characteristics as degree 
level, field of study, type of certification, 
and professional experience match the 
responsibilities to which they are assigned? 

Differences in public versus private schools. 
What are the differences in the patterns of 
resource allocation and utilization in public 
versus private schools? How do class 
sizes compare? How do the qualifications 



and compensation of staff compare? What 
factors underlie the differences in the 
quantities and characteristics of personnel 
resources in public and private schools? 

Why SASS Versus the CCD-fiscal Reports? 

For years, school finance researchers have 
paid considerable attention to the equity 
with which educational resources and 
services are distributed among local 
schools and districts. The problem with 
these studies is that they have focused on 
fiscal measures of resources which, 
because of geographic or inflationary cost 
differences, make it difficult to sort out the 
real differences in the levels of resources 
across schools and districts. NCES is 
currently organized in such a way that 
creates a rather narrow view of what 
constitutes school finance research. The 
division of NCES within which the 
Common Core of Data resides is 
responsible for the school finance agenda 
of the entire agency. The phrase “school 
finance” itself engenders an image of 
dollar allocations. The problem is that 
these dollar images provide little 
information about differences in the “real” 
resources devoted to education. Spending 
differences over time or across geographic 
regions reflect both real differences in 
resources as well as differences in the 
prices of comparable resources. 

Another significant problem in fiscal data 
is that it is ultimately organized according 
to reporting standards that differ across 
states and over time. Although NCES 
does publish an accounting handbook that 
provides standards for reporting fiscal 
data, not all states or local jurisdictions 
use the handbook. Moreover, those 
jurisdictions that do, do not necessarily 
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hold strictly to the accounting standards. 

In other words, there is a great deal of 
variability in the standards of reporting 
fiscal data among local jurisdictions. 
Analysis of resource allocation must 
ultimately rely upon more detailed and 
precise information on the physical 
ingredients utilized in the process of 
producing educational services. 

In addition, there is much discussion from 
NCES as well as the community of school 
finance researchers about moving to 
school-level fiscal analysis and about 
improving fiscal reporting to obtain 
estimates of how much schools are 
spending on different programs. SASS 
may offer the best opportunity to obtain 
good data on costs of programs at the 
school level through improvements in data 
collection on staff. SASS already collects 
a significant amount of important 
information on school personnel. A 
reorganization and reconfiguration of 
existing questions could enhance 
significantly the value and 
comprehensiveness of the data (specifics 
are offered below). SASS provides data 
that are representative within states as well 
as across the nation. Representativeness of 
SASS within states permits comparisons of 
differences in patterns of resource 
allocation that might be a result of 
differences in the administrative, 
regulatory, and fiscal environment within 
which schools operate across states. In 
addition to the added compatibility of 
information across states and local 
jurisdictions, using more direct measures 
of ingredients in this way allows increased 
flexibility to reorganize the data more 
easily for different purposes. 



Another issue confronted by NCES in 
considering whether or not to begin 
gathering school level data is the cost of 
such a move. Does one gather data on all 
schools across the country or a just a 
sample of schools? To gather fiscal data 
on all schools requires imposition of a 
rigorous set of standards for reporting 
fiscal information. SASS provides a 
cheaper alternative by gathering data on a 
sample of schools, and it has the potential 
for providing raw data in a more 
compatible format than is common in 
fiscal reporting systems. 

Many of the large-scale studies undertaken 
by the U.S. Department of Education 
every year could benefit by a somewhat 
expanded version of the SASS. These 
large-scale studies require significant data 
collection activities that to some degree 
duplicate information that already exists 
within SASS. With some expansion of 
SASS and some consultation with those 
government program officers involved in 
the design of these large-scale studies, the 
costs of data collection for these other 
studies might be reduced by virtue of the 
availability of SASS. One role of NCES 
should be to provide the kinds of databases 
that permit analysts to conduct these kinds 
of large-scale studies of issues related to 
the allocation of resources and to make 
these data available for use in the conduct 
of studies being required by other divisions 
of the U.S. Department of Education. 
Rather than having each study go off in a 
completely independent direction for data 
collection, the data for the SASS schools 
and districts and states could be used as a 
foundation for analysis. The advantage 
would be in efficiency in data collection, 
and it would provide larger samples of 
schools and districts for analysis for many 
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studies that currently rely on small samples 
of schools because of limited budgets. 

For example, one could easily envision 
ways in which the SASS data collection 
could serve as a foundation for analysis of 
resources utilized for special needs 
populations including Title I, special 
education, limited-English proficient, 
gifted and talented, vocational, and early 
education programs. With these issues in 
mind, the following recommendations for 
revising SASS will be elaborated upon 
below. 

• Staff should be reported through 
FTEs rather than full-time and part- 
time head counts 

• Staff should be reported according 
to types of delivery systems and 
programs they serve 

• All staff should be covered in the 
survey: both school-level staff in 
the sample schools and district-level 
staff in the sample districts 

• More information on individual 
staff should be gathered to improve 
data on geographic cost differences 
and inflationary trends-with ability 
to control for personal 
characteristics 

• Individual student information 
should be revised to obtain a more 
precise profile of needs and 
services 

• SASS data and samples should be 
more effectively linked with other 
NCES data such as the CCD fiscal 



files and with data on student 
outcomes 

• If it is necessary to sacrifice 

elements of SASS in order to make 
some of the revisions being 
recommended in this paper, then it 
is suggested that SASS reduce or 
eliminate the subjective elements on 
teacher attitudes and perceptions in 
favor of more objective data 

Redesigning SASS to Gather Data 
on School Resources 

Quantities of School and District Personnel 

The current SASS school questionnaires 
request information on the numbers of all 
full-time and part-time staff employed at 
the school. No such data on district level 
staff are currently available in the SASS 
battery of questionnaires. Two specific 
changes in the way school level data are 
reported could improve the quality of the 
information for understanding issues 
related to resource allocation. First, 
report personnel counts according to full- 
time-equivalents (i.e., FTEs) or in a way 
that permits translation into the intensity of 
services received by the school. Second, 
personnel should be broken down by major 
program category. A sample of the way 
in which such personnel data might be 
gathered is presented in the appendix to 
this report. The proposed changes are 
discussed in more detail below. 

Reporting FTE personnel . As an alternative 
to gathering the data on head counts of 
part-time and full-time staff positions, 

(e.g., as in items 16 and 17 of the 1993-94 
Public School Questionnaire ), these data 



should be reported in terms of FTEs (full- 
time equivalencies) or hours per week of 
services by job title of personnel. 
Certificated school personnel should be 
reported in terms of FTEs because their 
jobs generally are defined according to 
responsibilities for providing certain 
categories of services and less in terms of 
the time they spend providing them. Non- 
certificated school personnel should be 
reported in terms of total hours per week 
of services rather than FTEs or head 
counts. 

There are a number of ways in which this 
might be accomplished. For example, 
instead of just recording the number of 
full-time and part-time “school 
counselors,” the questionnaire could 
request the FTE number of counselors 
employed at the school. This FTE count 
should reflect only the time of the 
counselor devoted to a particular school. 
That is, a counselor who is a full-time 
employee but who works half-time at the 
school for which the questionnaire is being 
completed would be reported as a 0.5 FTE 
counselor. An alternative approach would 
be to maintain two columns in which the 
number of full-time staff by job title and 
the total FTE or total hours per week of 
part-time staff are recorded. 

Non-certificated staff are reported 
according to the total hours per week of 
service because there is a wide variation in 
the way districts define what constitutes a 
full-time non-certificated employee. In 
some cases, one FTE may be defined as an 
eight-hour day, five days per week, and 52 
weeks per year or a total of 2,080 hours of 
work per year, while in other cases one 
FTE might be different for each category 
of worker. For example, one FTE aide 



might be defined as six hours per day for 
181-day school year, while one FTE 
custodian could be defined as eight hours 
per day for a 210-day work year. 

Because of these potential complications in 
the way districts define FTE, one needs to 
gather information on non-certificated staff 
in a somewhat different fashion to ensure 
compatibility. For example, if a school 
employed two full-time custodians each 
working 40 hours per week and one half- 
time custodian working 20 hours per 
week, the total reported in the cell in 
Table 1 for custodians would be 100 hours 
per week. If custodians typically worked a 
200-day school year, then the total of 200 
days would be reported in the column 
labeled, “Typical contract days per year of 
work for this category of worker. ” 

Reporting staff by program. There is 
significant interest in the allocation of 
resources among programs for special need 
populations. As shown in Table 1, the 
school staffing data gathered on the SASS 
Public School Questionnaire could be 
reconfigured to gather the FTE and hours 
per week staffing data by program. 

Rather than simply reporting totals for 
each category of personnel as in the 
current SASS questionnaires, reporting 
could be done for the major school 
programs such as regular education, 
special education, the Title I program, 
programs for limited-English proficient 
students, and early childhood education 
programs. Even if the decision is made to 
count the number of full-time and part- 
time staff, a matrix of job titles by 
program categories could be reported to 
help assess the level of resources being 
utilized by special need versus the regular 
education program. 
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These data should be reported according to 
how people allocate their time and not 
according to how their salaries are paid. 
That is, an individual who is regarded as 
working in the special education program 
should be allocated to that program 
regardless of whether his/her salary is paid 
by the program. 

The reader should notice the job titles or 
categories listed in Table 1 represent a 
somewhat expanded list from the staffing 
categories used in the 1993-94 SASS 
school questionnaire. First, student 
support staff like psychologists and 
therapists have been separated into two 
categories. The reason for this is to 
capture more accurately the types of 
resources being utilized in certain special 
need versus regular programs. Therapists 
often provide direct services to students on 
a weekly basis, while psychologists and 
social workers tend to provide services 
more on an “as needed” basis. 

Second, teachers represent the most 
important resource in schools. For this 
reason, it is important to distinguish the 
use of teachers in primary as opposed to 
supplemental assignments. The self- 
contained classroom or departmentalized 
classroom teachers represent the vast 
majority of teaching staff within a school. 
However, in addition, many schools 
employ resource teachers or subject area 
specialists to provide supplemental 
instructional services to students. It is 
important to capture such differences 
because these different types of teaching 
personnel may have fundamentally 
different impacts on student outcomes. 

Finally, because of the significant interest 
in inclusionary or integrated programs, 



particularly for special education students, 
it is useful to break down resource 
teachers according to whether they are 
providing the traditional pull out or 
departmentalized programs as opposed to 
the more integrated services within the 
regular classroom. 

Custodial and maintenance staff have been 
separated from food service staff to 
capture the level of staffing of these 
programs. Food service staff may tend to 
vary greatly among schools depending on 
the way food production services are 
organized. In some cases, school lunches 
are produced in one central kitchen and 
delivered to other schools, while in other 
instances, each school may run its own 
cafeteria. To combine food and 
custodial/maintenance services would 
obfuscate the different ways in which these 
services are allocated and provided among 
schools. 

Reporting district administration and support 
staff by program. The increased interest in 
school-level resource analysis has led to an 
increased demand for information on the 
staffing of district administrative and 
support functions. There is already a fair 
level of detail about what schools look 
like, but little is known from SASS about 
what district administrative structures look 
like. Such data would be useful in 
assessing what kinds of functions are 
commonly performed by district as 
opposed to school-level staff. In the same 
way SASS could request data at the school 
level, it could also ask for data on 
personnel at the district level who do not 
show up on the school surveys. Staff who 
are assigned to schools and who provide 
services, even part-time, on a regularly 
scheduled basis at the school site should be 



recorded at the school, while all other 
personnel should be reported as working 
out of the district office. Once again, as 
with the school counts, personnel should 
be counted according to FTEs and should 
be allocated among the same program 
categories as suggested above for the 
school-level allocation. 

Characteristics of School and District 
Personnel and Compensation 

The SASS surveys have contributed 
significantly to the analysis of teacher 
supply and demand. These data have been 
used to describe in great detail the 
characteristics of teachers, to examine 
those who are leaving the teaching 
profession, and to analyze patterns of 
teaching assignments, to explore the 
factors underlying variations in teacher 
turnover, and to explain variations in 
salaries of teachers. In a current project 
for NCES, the SASS databases for 1987- 
88, 1990-91, and 1993-94 are being used 
to construct a geographic cost index and an 
estimate of inflationary trends in the costs 
of teachers. Development of such 
geographic or time-series cost adjustments 
requires detailed information on the 
compensation and personal characteristics 
of teachers and the schools, districts, and 
regions in which they work and live. 

These indexes may be used by NCES to 
adjust reports of nominal expenditures to 
reflect real differences in resources 
devoted to the provision of educational 
services. These indexes may be offered to 
funding agencies at the state or federal 
level to be used in adjusting distributions 
to reflect real purchasing power of state or 
federal aid to local or state jurisdictions. 
Such indexes provide policymakers with 
important information about the ultimate 



impact of changes in economic, 
demographic, and political trends on the 
supply of teachers’ services and the costs 
of educational services across local 
jurisdictions and over time. 

In order to improve upon the analysis 
being done in the development of models 
of teacher compensation, there are three 
areas in which the SASS questionnaires 
need to be enhanced: the measurement of 
teacher quality, the measurement of 
benefits, and the extension of individual 
data to include other categories of 
personnel. Each of these is discussed 
further below. 

Teacher characteristics. One of the most 
contentious areas in the analysis of teacher 
supply and demand involves attempts to 
measure teacher quality. It is an illusive 
concept and one that defies any simple 
approach to measurement. The best that 
one can hope to do is to obtain information 
on a comprehensive set of teacher 
characteristics so that one can explore the 
patterns of variation across schools and 
districts, and one can associate those 
differences with variations in compensation 
and productivity. In 1987-88, the SASS 
teacher questionnaires requested that 
individuals record the colleges or 
universities attended to obtain their 
degrees. This question was dropped in 
1990-91 and was restored in 1993-94. 

This variable is one possible indicator of 
the intellectual capability of teachers in 
that it permits one to match teachers to the 
selectivity (e.g., as measured by the 
average Scholastic Aptitude Test (SAT) 
scores of entering freshmen) of the 
colleges in which they were trained. The 
presumption is that individuals graduating 
from more selective institutions of higher 
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education may have a greater capacity to 
become quality teachers. It is important 
that this question not be dropped in the 
future as it is one of the few direct 
attributes of teaching personnel that is not 
explicitly recognized in public school 
district salary schedules. It also would be 
useful if NCES would, as a matter of 
course, obtain data on the characteristics 
of colleges (e.g., student selectivity, 
acceptance ratios, percent of students from 
the top ten percent of the graduating high 
school class) that would be matched to this 
database prior to making it available to the 
public. 

It would be preferable to obtain more 
direct measures of teacher intellectual 
capabilities. Teachers may well be in any 
place in the distribution of SAT scores 
within a particular college or university. 

It might be worthwhile for NCES to 
explore the possibilities for obtaining 
information on the SAT scores of the 
individual teachers. One might anticipate 
that teachers would not remember their 
actual scores, but they might remember the 
range within which they scored on the 
math and verbal components of the SATs. 
This might be asked in terms of raw score 
(e.g., below 400, 400 to 499, 500 to 599, 
600-699, above 700) or percentile rankings 
(e.g., below 50 percentile, 50 to 75 
percentile, 75 to 90 percentile, or above 
90 percentile). An alternative might be to 
explore the possibility of merging records 
obtained from the Education Testing 
Service. 

Fringe benefits. Data on the costs of 
benefits for various categories of personnel 
would be important in improving the 
quality of the information on 
compensation. The current SASS 



questionnaires ask teachers what kinds of 
benefits beyond salary compensation they 
receive from the public and private school 
systems in which they teach. However, 
there is no indication as to the value or 
cost of these benefits. It is important in 
order to adequately characterize personnel 
compensation that one knows how much 
money is contributed by the school (in the 
case of private schools) or district (in the 
case of public schools) on behalf of each 
employee for health and welfare insurance 
premiums, the retirement plan, and other 
categories of personnel. Using this 
information, one can estimate the total cost 
of compensation (i.e., salaries plus the 
cost of benefits) for each teacher. 

Conceptually the importance of fringe 
benefit information is to recognize that 
schools and school systems use a 
combination of salary and benefits to 
attract teachers. In the long run, one 
would expect teachers to sort themselves 
out across districts in such a way that a 
dollar of fringe benefits (in terms of the 
cost to the district) is equal to a dollar of 
salary. That is, the individual would trade 
off salary to receive fringes up to the point 
at which the two have equal relative 
marginal value in comparison to their 
relative cost to the district (or school). If 
this is true, then one should be able to add 
salary costs and the costs to the district of 
fringe benefits together for purposes of 
analyzing compensation. 

Much analysis has been done of variations 
in salaries, but only limited analyses have 
ever been done on full compensation of 
salaries and fringe benefits together. With 
such information, researchers could 
examine the impact of including fringe 
benefits in the analysis of teacher 
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compensation. This would represent a 
major step forward in examining the 
patterns of differences in teachers’ salaries 
across geographic locations and between 
public and private schools. Indeed, the 
data suggest that the differences in public 
and private school teacher compensation is 
larger than the differences between public 
and private school teacher salaries because 
of the lower levels of benefits offered by 
private schools. Moreover, how does the 
introduction of the benefit data affect the 
patterns of variation in the costs of teacher 
services across different geographic 
locations? 

Part of the reason that fringe benefit data 
have not been gathered in the past is that 
these data are difficult to obtain because of 
the complexity of the benefit packages 
across employers. Many districts offer 
multiple health plans from which teachers 
may choose and the premium paid by the 
district for these plans varies with the size 
of the employee’s family. Moreover, most 
individual employees probably do not 
know exactly what the district 
contributions are without substantial effort. 
While there is probably no way to get 
individual information for samples of 
school personnel on benefits, one could 
obtain from the district-level questionnaires 
in public schools and school-level 
questionnaires for private schools some 
information on the structure of benefits 
that would allow researchers to estimate 
the value or cost of a benefit package. 

Benefits come in two basic forms: those 
that are lump-sum payments per employee 
and those that are a percentage of salary. 
Health and welfare benefits are most often 
in the form of insurance premiums paid by 
the employer on behalf of the employee, 



and the amount per employee is fixed for 
any given benefit plan. This amount can 
be represented as a fixed, lump-sum 
payment per employee. These health and 
welfare benefits include basic health 
insurance, major medical plans, dental 
insurance, vision insurance, and life 
insurance. To obtain estimates of the costs 
of these plans, the SASS would include a 
series of questions on the public school 
district or private school questionnaires 
regarding the lump sum payments paid on 
behalf of teachers or certificated school 
employees for each category of benefit for 
the most widely used plan (assuming the 
district offered multiple plans): basic 
health, major medical, dental, vision, or 
life insurance. 

Another category of benefit is most 
commonly paid as a percentage of salary 
of the employee. These benefit elements 
include, for example, contributions by the 
employer to retirement plans (e.g., public 
employee pension plans), social security 
payments, unemployment compensation, 
workers’ compensation plans, and 
disability insurance. For example, the 
district might contribute some fixed 
percentage of the employee’s salary to a 
retirement program. In many states, 
teachers become a part of a public 
employees’ retirement program established 
for certificated school employees. In some 
states, teachers may be part of the social 
security system. Unemployment 
compensation and workers’ compensation 
programs are most often based on the 
experience of the district and rates paid by 
districts on behalf of employees will vary. 

A further complicating factor in obtaining 
information on employee benefits is that in 
some instances, states provide for the 
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direct payment of benefits of public school 
personnel. For example, the state of New 
York makes direct payments to teacher 
retirement systems for all teachers in the 
state. The Department of Education in 
Kentucky provides direct payments on 
behalf of public school employees for 
many of the elements of the benefit plans. 
The problem is that district administrative 
staff who are being asked to provide these 
data for a district-level questionnaire may 
not have the information on the cost of 
these plans per employee. 

While benefits can be complex, they 
nevertheless represent an important 
component of compensation, and one on 
which some data can be obtained. Data 
from individual districts can be used to 
estimate benefit rates for individual 
employees. To illustrate one approach to 
obtaining information on benefits, a 
sample set of questions is included in the 
appendix to this paper. 

Samples of other school personnel. 

Currently, SASS includes samples of 
individual teachers, principals, and 
professional library media personnel. 

These data may be used to examine issues 
related to the distribution of personal 
characteristics and abilities as well as the 
job assignment attributes of critical 
members of school district staff. If NCES 
were to consider improvements in SASS 
that would ultimately permit analyses of 
educational productivity, it would be of 
critical importance also to include more 
extensive information about the personal 
characteristics and job assignments of 
other school and district staff who come 
into contact with children or who are 
involved in leadership activities in schools 
and districts. Data on other school and 



district staff would also be extremely 
valuable in enhancing the value of SASS 
for analysis of the supply and demand for 
other critical school personnel and for 
developing of geographic or inflationary 
cost adjustments. 

Specifically, there are two other important 
categories of personnel that could be 
covered in these samples. First, 
instructional aides play a potentially 
important role in instructional services in a 
number of school programs. Instructional 
aides are used to supplement the 
instructional program for students with 
special needs (e.g., students who have 
limited-English proficiencies or who are 
eligible for special education services). 
They are also used to increase the ratio of 
adults to students in regular education 
classrooms. Data on samples of 
instructional aides could provide 
policymakers with a better look at the 
ways in which aides are being utilized in 
schools, the qualifications of these 
individuals, and the patterns of 
compensation. 

While SASS currently gathers information 
on principals, there is virtually no 
information on the high-level 
administrative staff (e.g., superintendents, 
deputy superintendents, and other program 
administrators) of school systems. Who 
are these individuals leading the nation’s 
schools? What kinds of qualifications do 
they bring to these jobs? What factors 
underlie the patterns of variations in 
compensation of high-level administrators 
in school districts? 

It would also be useful to gather data on 
samples of other noncertificated school 
employees. These data could be used in 
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conjunction with FTE counts of staff to 
estimate costs at the school and district 
level. The data on individual school 
personnel could be used to explore the 
patterns of variation in rates of pay over 
time or across geographic locations and 
their implications for variations in the 
costs of educational services. Currently, 
the only such data available to researchers 
studying these costs are from the samples 
maintained by the Current Population 
Surveys. Alternatively, SASS could gather 
data on average salary levels or hourly 
rates for those categories of personnel for 
which no individual data are provided. 

Data on State Environment in Which 
Schools Operate 

As suggested earlier, SASS could enhance 
the ability of researchers to examine the 
effects of state policies on resource 
allocation and outcomes by systematically 
gathering data on state policies in the same 
year as the SASS questionnaires are 
administered to local schools and school 
systems. In each year of SASS, surveys 
could be sent to the state departments of 
education to gather some of the following 
kinds of information: 

Funding formulas . Information describing 
the structure of the school finance system 
in place within the state. This would 
include not only the structure of the basic 
distribution formula for elementary- 
secondary education, but also the nature 
and structure of the categorical program 
funding that currently exists within the 
state. What kinds of formulas are used: 
flat grants, foundation formulas, 
percentage equalizing, guaranteed tax base, 
or district power equalizing? What do the 
formulas look like? Are there caps or 



limitations on local contributions to 
educational spending? How much does the 
district contribute to categorical programs 
for special needs children? How are the 
categorical formulas connected with 
regular education funding? Are there 
limitations on the percentage of children 
who can be declared eligible for certain 
special needs programs (e.g., special 
education)? Are funds targeted to these 
populations? 

Employee benefits. Information on benefit 
programs for public employees could be 
gathered to supplement the benefit 
information obtained at the district level. 
What contributions does the state make on 
behalf of teachers to benefits such as 
health and welfare insurance or retirement 
plans? 

Public school regulations. Information on 
the regulations under which public schools 
operate could be gathered to allow 
comparisons across states of the impact on 
the way resources are allocated. What 
kinds of curricular standards exist within 
the state? What is the structure of teacher 
certification within the state? What kinds 
of laws or regulations exist with regard to 
collective bargaining agreements for 
certificated employees? 

Private school regulations. Information on 
the nature of private school regulations 
could be gathered. Under what kinds of 
regulations do private schools operate 
within each state? Are private school 
teachers required to be certified? What 
kinds of safety standards are imposed on 
private schools? What kinds of regulation 
of private school curriculum exists, if any? 
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While currently there exist other data 
collection besides SASS on some of these 
elements, it would enhance the value of 
these data significantly if they could be 
linked to the SASS data collection activity. 

Data on Individual Students 

The recent addition to SASS of 
information on individual students provides 
a useful structure for obtaining data on the 
types and costs of services received by 
individual students across the nation. The 
current survey provides a listing of some 
of the classes taken by the student as well 
as whether or not the student is receiving 
certain other services. However, the 
survey falls short of providing sufficient 
information to estimate costs of these 
direct services. The importance of these 
individual student data is that they have the 
potential to allow researchers to explore 
the patterns of variation in expenditures 
across students with differing 
characteristics or educational needs. 

Over the years, there have been numerous 
attempts at the school level to determine 
the expenditures on programs for special 
needs students (e.g., special education). 

But it has been more difficult to obtain 
estimates of the costs of serving a 
particular type of special needs student. 
Data are needed at two levels. At one 
level, policymakers want to know how 
much is being spent to provide certain 
types of services and programs to certain 
categories of students. This type of 
information can be obtained at the 
institutional level (e.g., school- or district- 
level data collections). Based on such 
institutional data, one can obtain estimates 
of the average expenditures on specific 
categories of services such as the special 



day class, a resource (pull out ) program, 
or a related service such as speech 
therapy. For example, how much is being 
spent on special education services? How 
much is being spent on special day classes 
or resource programs and how many 
students receive such services? 

However, such institutional data do not 
indicate how these services are combined 
to meet the needs of specific categories of 
children ? They do not provide 
information on the full cost of educating a 
particular type of child since most children 
are not served within a single program. 

For example, the vast majority of special 
education or other special needs children 
receive services in the regular program. 
What is required is information on 
individual children that indicates the types 
of services received, the amounts of time 
spent receiving those services, the staffing 
of those services, and the size of the 
classes or caseloads within which those 
services are delivered. Specifically, one 
would need the following information to 
enable researchers to estimate costs. 

Range of services received. The current 
survey provides an overview of the types 
of programs in which the child participates 
and some indication of some of the special 
services received. Using a structure 
similar to that used to record teachers’ 
schedules on the teacher questionnaire, the 
student records questionnaire of SASS 
could provide for coding of all of the 
instructional and related service 
assignments of the student. The 
assignments could include a listing of all 
of the secondary courses or classes as well 
as elementary placements including self- 
contained classroom or resource pull out 
programs. 
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The student’s time. For each instructional 
or related service assignment, the amount 
of time (e.g., hours or minutes per week) 
the child spends in each assignment could 
be recorded. This information could be 
used to determine: How do children spend 
their time in school? How much time do 
they spend in the regular classroom? How 
much time is spent in supplemental or 
resource programs designed for special 
needs populations? 

Staffing. For each instructional or related 
service assignment, one could record the 
types and time spent by various types of 
staff members while the student was 
present. For example, if the student was 
enrolled in a self-contained classroom for 
30 hours per week, the respondent would 
record the number of hours the following 
staff were present in the classroom: the 
regular classroom teacher, a special 
education resource teacher, an instructional 
aide, or a personal aide for this child. 

Class or group size. For each instructional 
or related service assignment, one could 
record the size of the class or group (e.g., 
for a pull out session) in which the child 
was served. 

These kinds of data could provide valuable 
information on the patterns of direct 
services received by students across the 
country. Administrative and support 
services expenditures could be ascertained 
from the school- and district-level data if 
FTE counts are obtained as suggested 
above. The information on direct services 
would allow researchers to address the 
following kinds of questions: 



• How many hours do children spend 
in school? How do they spend 
their time in school? 

• What kinds of courses are being 
taken by students and how do these 
differ according to the educational 
needs of students? 

• What percentage of students are 
being served in different special 
needs programs around the 
country? 

• How do the ways in which these 
children are served differ across 
communities categorized by 
urbanicity, size of district, state, 
and household income or poverty? 

• What are the expenditures for 
different categories of children 
categorized by disability or other 
special need characteristics? 

If one were able to take the next step in 
this data collection of obtaining outcomes 
for individual students, then the 
possibilities for beginning to unravel the 
mysteries of input/output relations in 
education would be enhanced significantly. 

Linking to Other Databases for Analysis 

A key to making NCES databases valuable 
lies in the ability to link the various 
elements. SASS provides us with the 
opportunity of gathering detailed and 
valuable data on the ingredients of 
educational production and services as 
reflected in the staffing patterns across 
local schools and districts. But this is only 
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the first step. Another piece of the puzzle 
is in linking fiscal data collection activities 
to the staffing information. If the 
Common Core of Data (CCD) samples or 
universe files could be linked with the 
SASS data collection, each of the two 
activities would be enhanced. Although it 
would be valuable to gather information on 
other nonpersonnel ingredients in the 
educational process through SASS, it may 
simply be creating too much in additional 
burden of data collection to obtain any 
additional information beyond the staffing 
information previously suggested. 

However, if the CCD fiscal files could be 
obtained from the schools and districts 
included in the SASS data collection 
activity, this linkage would provide 
additional information about some of the 
nonpersonnel resources provided to 
students. Combining the staffing 
information with the fiscal data on 
nonpersonnel expenditures would enhance 
the value of both the staffing and fiscal 
data. 

An additional and significant step would be 
to link SASS with data on student 
outcomes, either the outcomes of 
individual students included in the sample 
of students for whom the Student Records 
Questionnaire is obtained or the outcomes 
for the SASS schools. This kind of 
linkage would allow researchers to explore 
productivity issues. How do outcomes 
relate to the quantities and qualities of 
staffing in different schools? How do 
these relationships differ in schools serving 
students with exhibiting differing 
educational needs or economic 
circumstances? 



Burdens of Data Collection: What Do We 
Give Up in SASS? 

The current SASS questionnaires contain a 
combination of objective and subjective 
data items. Among the objective items are 
counts and characteristics of staff. Among 
the subjective items are questions about the 
attitudes and perceptions of teachers, 
principals, and library media specialists. 
Perhaps SASS should focus as much as 
possible on gathering data on the more 
objective measures of what is going on in 
schools and less on the subjective 
measures. This is not to say that 
attitudinal measures are unimportant. The 
problem is that attitudinal issues are 
difficult to benchmark and the data are 
difficult to interpret. What one person 
designates as a serious problem, another 
person might designate as a moderate or 
even a minor problem. Ultimately, 
objective measures of events, activities, 
and behaviors and their relationship 
between one another, or with schooling 
outcomes, permits determination through 
analysis when a particular event, activity, 
or behavior is a serious problem. 

For example, the questionnaires ask about 
teacher and principal perceptions about the 
seriousness of certain kinds of problems in 
the school such as student tardiness, 
student absenteeism, teacher absenteeism, 
students cutting class, physical conflicts 
among students, and robbery or theft 
among many others. It may be preferable 
to obtain objective counts of events or 
behaviors that would reveal the degree to 
which students were tardy, absent, cut 
class, or were involved in incidents or acts 
of physical violence or robbery that were 
reported to the school office. 



The purpose of this is not to suggest that 
attitudinal data are of no consequence or 
value. First, in the absence of more 
objective measures of the events, 
activities, or behaviors that these 
subjective measures attempt to document, 
such subjective measures may be the only 
source of information. However, what is 
being argued is that where possible, 
objective measures should be requested 
rather than information on attitudes and 
perceptions. If schools do not maintain 
reporting mechanisms on certain student 
behaviors or activities, then one might 
question whether or not the activity is 
really a serious problem. 

Second, one of the questions posed in the 
design of this series of papers is what 
elements in SASS could be sacrificed in 
order to make room for the additional data 
collection being proposed in this paper. In 
the opinion of this author, if something 
needs to be sacrificed in order to make 
room for a revised data collection strategy, 
it is the questions on the subjective 
perceptions and attitudes of school 
personnel. 

Summary and Conclusions 

1. Could information on the source, level, 
and allocation of such resources be valuable 
to researchers and other data users ? 

With the continuing interest in issues 
related to educational productivity, it is 
going to be essential for researchers and 
policymakers to understand the 
relationships between resource allocation 
and outcomes in U.S. schools. Some 



redesign of SASS with attention paid to 
linking SASS with other important 
databases could be of tremendous value to 
the research and policy community in 
education in exploring these issues of 
educational productivity. 

2. Could collection of such data he 
reasonably integrated into current SASS 
collections without increasing overall 
respondent burden ? 

SASS already gathers the kinds of data that 
are important for understanding these 
patterns of resource allocation. A number 
of the suggestions made in this paper for 
the redesign of SASS involve asking for 
the same type of data in a slightly more 
detailed and reorganized form. For 
example, this paper suggests 
reconfiguration of the data collection on 
staff to be done on the basis of full-time 
equivalencies and to gather staffing 
information by program of service. 

There has been some discussion at NCES 
of gathering fiscal data at the school site. 
This paper suggests that it may be more 
efficient to gather resource data through a 
vehicle such as SASS where the physical 
ingredients of the educational production 
process can be measured explicitly. 
Gathering fiscal data may be more 
burdensome in that it requires 
reconfiguration of fiscal data to match the 
categories (i.e., objects, functions, and 
programs) of expenditure being requested. 
Physical ingredients (e.g., FTEs) such as 
individuals by job title and program are 
more easily counted. Quality data on 
personnel at the school level gathered 
through SASS could reduce the need for 
such detailed fiscal information. 



3. What would need to be given up to 
accommodate this increase ? 

It was suggested that if, indeed, something 
must be given up to accommodate the 
suggested revisions or additions to SASS, 
the items focused on gathering the attitudes 
and perceptions of teachers and principals 
should be sacrificed. Attitudinal data are 
more difficult to interpret than objective 
counts of events or occurrences, and in 
most instances, the attitudes are simply a 
way of trying to obtain information about 
things that can more properly and usefully 
be measured by objective data (e.g., 
student tardiness or absenteeism). If such 
information is important, it was argued in 
this paper that the attitudinal data be 
eliminated in favor of more objective 
measures of events, activities, and 
behaviors of students or staff. 

4. Is SASS the appropriate vehicle? 

What makes SASS the appropriate vehicle 
for gathering resource data is that it 
already focuses a significant amount of 
attention on this kind of information and it 
is focused at the school level. The 
recommendations made in this paper 
involve modifications or additions that fit 
well within the existing framework and 
structure of the SASS questionnaires. 
Moreover, where expansion of the SASS 
structure is being recommended, it is for 
the purpose of expanding in important and 
significant ways the kinds of analyses that 
can be carried out using SASS. For 
example, it was suggested that some of the 
existing data collection regarding state 
school finance systems be integrated with 
the SASS data collection so that the 
information could be more easily linked 
for analytical purposes. 



5. Is this information currently collected bv 
states. LEAs, or by other means ? 

There has been some discussion within 
NCES about the possibility of moving 
toward the collection of school-level fiscal 
data. The purpose of such school-level 
fiscal data is to obtain better information 
on the patterns of resource allocation as 
they relate to issues of equity and 
productivity in education. But because of 
the complexities of making appropriate 
adjustments in fiscal information for 
geographic or longitudinal input price 
differences, SASS would be a potentially 
more fruitful area of exploration for 
examining equity and productivity. SASS 
has the potential of gathering the physical 
ingredients rather than fiscal data that 
obscures real input levels. By gathering 
data on physical ingredients, SASS avoids 
many of the compatibility and definitional 
issues that surround the ways in which 
fiscal data are reported by local 
jurisdictions. Finally, using SASS as a 
source of resource data avoids the 
problems associated with disseminating 
and implementing a set of complex 
standards for reporting fiscal information 
that would be required for developing a 
school-level fiscal data collection system. 
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THE SCHOOLS AND STAFFING SURVEY FOR 1998-99: 
DESIGN RECOMMENDATIONS TO INFORM BROAD 
EDUCATION POLICY 



Erling E. Boe, University of Pennsylvania 



Introduction 

The Schools and Staffing Survey (SASS) 
and the Teacher Followup Survey (TFS), a 
longitudinal component of SASS given in 
the subsequent year, have been 
administered by the National Center for 
Education Statistics (NCES), the U.S. 
Department of Education, during three 
different time periods: 1987-89, 1990-92, 
and 1993-95. 1 Though NCES has made 
incremental improvements in SASS 2 
between administrations and supplemented 
the basic SASS with additional 
questionnaires addressing special topics 
such as Indian education and library 
issues, there has been intentionally a great 
deal of continuity in the content of the 
several SASS questionnaires to permit the 
study of trends over the six-year period of 
SASS. 

In preparation for the next administration 
of SASS/TFS scheduled for 1998-2000, 
NCES has decided to reassess the design 
of SASS, including questionnaire content 
and related matters such as the possible 
linking of SASS to other NCES surveys 
that collect student background and 
achievement data. This reassessment is 
subject to the constraints that (a) SASS 
will continue to be a cross-sectional survey 
based on national probability samples, (b) 
the focus of SASS will continue to be on 
schools, including their staffing, at the 



elementary and secondary levels, and (c) 
changes to the content of SASS should not 
increase the burden on respondents 
completing SASS questionnaires (i.e., any 
new content will have to be counter- 
balanced by selective deletions of old 
content). 

As a contribution to NCES’s current 
reassessment of the design of SASS, this 
paper will consider future questionnaire 
content in broad scope with respect to how 
SASS can best inform education policy 
issues pertaining to schools and their 
staffing— especially important issues that 
may emerge over the next two decades. 
Specifically, the objectives of this paper 
are to: 

1. Reconsider the goals, foci, and 
strategy of SASS, taking into 
account the original framework 
established in the mid-1980s and 
SASS’s potential to inform broad 
education policy in the future. 

2. Identify potential new areas of 
questionnaire content related to 
schooling that are likely to be of 
importance to education policy 
issues in the future. 

3. Review the content of the most 
recent (i.e., 1993-94) SASS 
questionnaires in light of (a) the 
existing balance between teacher 
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supply and demand content and 
other school content, (b) the extent 
of coverage of particular topics that 
appears to be excessive or the 
coverage of topics that appears to 
be of relatively low priority, and 
(c) redundancy across 
questionnaires. An outcome of this 
review will be the determination of 
questionnaire content that can be 
compressed or deleted to 
accommodate expansion in other 
content areas. 

4. Recommend priorities among 
potential new content areas and 
essential areas from past SASS 
administrations that might be 
included in the 1998-99 
administrations of SASS. The 
focus will be on factors determining 
the nature and quality of schooling 
that are amenable to education 
policy interventions, as 
distinguished from aspects of the 
broader social, economic, and 
political contexts that also shape the 
form and functions of education 
within society. 

Two caveats involved in the pursuit of 
these four objectives should be noted. 

First, the emphasis will be on schools in 
the public sector because policy formation 
in education applies predominantly to 
public schools. Second, data collected by 
SASS should be relevant to education 
policy assessment and decisions at the 
local, state, and federal levels. Even 
though most education policy is made at 
the local and state levels, the mix of local, 
state, and federal influences on schooling 
has been in gradual flux and is expected to 
continue to change. 



School Improvement Versus 
School Change 

In preparation for the fourth administration 
of SASS in 1998-99, NCES is 
reconsidering the value of continuing to 
emphasize teacher supply and demand data 
in contrast with other data that may 
contribute more toward school 
improvement. Specifically, NCES asked, 
in commissioning this paper, "What 
nationally representative schools and 
school process data will inform our 
thinking about and work toward improving 
schools in the next 10 to 20 years" 
(emphasis added). Given this task, 
possible changes in the goals, foci, and 
strategy of SASS will be considered in 
light of their potential contribution to 
improving schools. 

If the value of SASS is to be measured by 
its contributions to school improvement, it 
is necessary to define what is meant by 
school improvement. To begin, 
distinctions must be made among (a) 
changes in policies that are designed to 
improve schools, (b) changes in programs 
and practices that are designed to improve 
schools, and (c) changes in school 
performance reflected in indicators such as 
outcome measures (e.g., higher 
achievement test scores), resource 
efficiency measures, school climate 
measures, and approval ratings by 
stakeholders (e.g., parents, the public). 
Ordinarily: 

• Changes in policy are expected to 
result in changes in programs and 
practices that conform to the new 
or revised policies 
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• Changes in programs and practices, 
in turn, are expected to result in 
improved school performance 

It is well known, however, that (a) 
changes in policies (ordinarily construed 
as, and intended to be, improvements) do 
not necessarily result in changes in 
programs and practices, and (b) changes in 
programs or practices (often construed as, 
and intended to be, improvements) do not 
necessarily result in improved indicators of 
school performance. Therefore, the intent 
of policymakers and educators to improve 
policies, programs, and practices by 
implementing changes is not certain to 
reap the desired effects on school 
performance. Whether such changes 
actually affect school performance is an 
empirical matter, subject to systematic 
measurement such as by standardized 
achievement tests. 

Furthermore, there is even a problem in 
determining what actions or conditions 
constitute "school improvement" because 
this involves judgment. What some regard 
as an improved policy, program, or 
practice, others may view as a reversal. 

For example, some view private school 
vouchers as a promising policy reform, 
while others regard them as a basic threat 
to public schools (Jennings & Stark, 

1995). In contrast, there is often 
consensus about what changes in school 
performance represent improvement, such 
as rising academic achievement scores. 
Even so, judgments differ about whether 
some changes in school performance 
represent improvement— as evidenced by 
the conflict in several states over 
outcomes-based education. 



The upshot of this line of thinking is that it 
will be more useful and productive to 
translate the concept of "school 
improvement" to that of "school change," 
and to determine whether changes in 
policies, programs, and practices actually 
lead to changes in school performance. 

Let others (policymakers, professional 
educators, the press, and the public) debate 
whether changes observed actually 
represent improvement. In contrast, SASS 
should be used to make major 
contributions to understanding various 
changes in the interlocking sequence of 
policies, programs, practices, and 
performances by establishing baseline 
status data and monitoring changes from 
these baselines over time. 3 Specifically, 
SASS can be used to: 

• Monitor changes in policy. With 
respect to policy changes, SASS 
has not monitored the status of, or 
changes in, federal and state 
education policy. However, SASS 
has and can continue to monitor the 
status of, and changes in, policies 
at the local education agency (LEA) 
level through the Teacher Demand 
and Shortage Questionnaire. One 
value of monitoring the status of, 
and changes in, LEA policies is to 
determine how policies at the 
federal and state levels are being 
interpreted and translated into local 
policy. Another value of 
monitoring LEA policies is to 
determine how well programs and 
practices at the school level 
conform to LEA policies. 

• Monitor changes in programs and 
practices. SASS has and can 
continue to monitor the status of, 
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and changes in, programs and 
practices at the LEA and school 
levels through the Teacher Demand 
and Shortage Questionnaire and the 
School Questionnaires. One value 
of monitoring the status of, and 
changes in, programs and practices 
is to determine how well they are 
conforming to federal, state, and 
local policies. Another value of 
monitoring the status of, and 
changes in, programs and practices 
is to measure many of the 
dimensions of schooling that impact 
on school performances. 

• Monitor changes in performance 
indicators. SASS has and can 
continue to monitor the baseline 
levels of several indicators of 
school performance, and changes 
from these baselines, through the 
School, Principal, and Teacher 
Questionnaires. 4 Though most 
school performance indicators are 
measured by instruments other than 
SASS, SASS nonetheless serves a 
special role in monitoring some 
school performance indicators (such 
as school climate) because SASS is 
the most direct, and possibly only, 
source of such national data 
available. 

Change in school policies, programs, 
practices, and performances is used here 
as a generic concept to include school 
improvement, reform, and restructuring, 
since all such initiatives entail various 
forms of change. Clearly, it is not 
possible to predict many of the school 
improvement, reform, or restructuring 
initiatives that will continue, or be 
initiated, in the coming two decades. Who 



could have predicted in 1983 the variety 
and form of such initiatives that were 
taken in the decade following publication 
of A Nation at Risk ? Thus, it will not be 
prudent to redesign SASS to monitor 
specific current reforms per se (such as 
systemic reform). Instead, SASS should 
monitor the fundamental dimensions of 
schooling that are amenable to 
manipulation by public policy for the 
purpose of improving school performance. 

Considering that SASS is an instrument for 
collecting basic data on schooling and is 
scheduled to be administered only every 
five years in the foreseeable future, it 
should be designed to cover essential, 
enduring aspects of schooling that will be 
of continuing importance, as distinguished 
from covering specific reforms or current 
narrow issues. For example, if SASS 
monitored the character of, implementation 
of, and changes in curriculum frameworks, 
student assessment, and teacher 
preparation (both pre- and inservice), it 
would be monitoring fundamental 
dimensions of schooling that, as a group, 
are basic components of systemic reform. 

In the event that systemic reform, as such, 
proves to be a passing fad within a few 
years, SASS will nonetheless have tracked 
three fundamental dimensions of schooling 
that are likely to be relevant to future 
educational policy. 

As noted previously, policy-based changes 
in such dimensions might be viewed by 
some as constructive reforms, and as 
reversals by others. Therefore, as a public 
agency dedicated to collecting and 
reporting unbiased statistical information, 
NCES should not appear to be an advocate 
for or against any reform. Furthermore, 
the publication of major reports of 
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information based on SASS data usually 
requires four years from the time of 
questionnaire design— all the more reason 
for SASS to concentrate on fundamental 
dimensions of schooling that will be of 
enduring interest instead of concentrating 
on any current high-profile reform 
initiative. 



The Goals, Foci, and Strategy of 
SASS; A Reconsideration 

Past SASS Goals. Foci, and Strategy 

Since its inception in 1987-88, the goals, 
foci, and strategy of SASS can be 
characterized as follows: 

• Goals'. The primary goal of SASS 
has been to provide data "that will 
contribute to the development of 
sound educational policies at all 
levels of government" (NCES, 

1994, p. 2). SASS has also served 
as a source of national and state 
data about schools and their staffing 
for educators and researchers. 

• Foci: SASS has focused primarily 
on the teaching force (K through 
12) and secondarily on school 
policies, programs, and 
administrators. Its distinctive 
strength has been the 
comprehensive data collected about 
teacher characteristics, 
qualifications, and attitudes, and 
about teacher demand, supply, 
turnover, and workplace conditions. 
These aspects of the teaching force 
are referred to collectively in this 
paper by the expression "teacher 
supply and demand. " 



® Strategy: The strategy adopted for 
SASS has been to collect detailed 
descriptive data about schools and 
their staffing, as distinguished from 
collecting data to test specific 
hypotheses or to evaluate specific 
policies, programs, or practices. 

This strategy is particularly suited 
to the sample survey method, and 
is in keeping with NCES’s mission 
to collect, analyze, and disseminate 
education statistics. 

It is these basic attributes of SASS that 
should be reconsidered in its redesign, 
along with the implications for 
questionnaire content. 

Future SASS Goals 

The goals of SASS should continue to be 
the collection of data that will contribute to 
the development of sound education 
policies at all levels of government and 
that will be useful to educators and 
researchers. Changes in SASS for 1998- 
99 should be made in its foci and strategy, 
and in questionnaire content that follow 
from decisions about foci and strategy. 

Future SASS Foci 

The fundamental question in redesigning 
SASS is whether the current primary focus 
on collecting data about teacher supply and 
demand should be maintained or modified. 
Major candidates for alternative foci are 
instruction, school governance, and school 
organization. In reconsidering the foci of 
SASS, the following guiding principles 
should be observed: 

® The importance of continuity in 
data collected in previous 



administrations of SASS should be 
recognized. In light of previous 
investments in establishing baseline 
data and changes over time in 
numerous dimensions fundamental 
to schooling, continuity should be 
valued highly. Therefore, the 
redesign of SASS should not start 
with a clean slate. 

SASS data that will be of maximum 
value to policymakers and others in 
describing basic dimensions of 
schooling and in monitoring change 
over time in these dimensions 
should be of high priority. In 
deciding what educational 
phenomena are sufficiently 
important to quantify, the following 
guidelines can be used: 

Select fundamental aspects of 
schooling that have been the subject 
of major policy action in the recent 
past, such as teacher preparation 
and qualifications, school 
accountability, decentralization of 
authority, deregulation, 
instructional technology, and the 
like. Such topics are likely to be of 
policy interest in the next decade or 
two. 

Select fundamental aspects of 
schooling that have been the subject 
of recent policy analysis and 
debate, but only of minor policy 
action to date, such as contracting 
for school management and 
instruction (i. e. , privatization), and 
vouchers for private school tuition. 
Some such topics are likely to 
attract major policy action in the 
future. 



Select aspects of schooling of major 
public concern such as instruction 
in basic skills, student discipline, 
and school safety. 

• The roles and contributions of other 
NCES surveys should be 
recognized, especially those that 
collect data about basic dimensions 
of schooling and monitor change in 
these dimensions. Even though 
certain data may be of critical 
importance, SASS should not 
ordinarily collect data that are 
available from other sources. 

These principles for reconsidering the foci 
of SASS will be used later in analyzing 
alternative emphases that might be 
employed for the design of the fourth 
administration of SASS. 

Future SASS Strategy 

In keeping with NCES’s mission to 
collect, analyze, and disseminate education 
statistics, the strategy adopted for SASS 
has been to collect detailed descriptive data 
about schools and their staffing. Without 
challenging NCES’s statutory mission, 
however, it is useful to examine three 
potential uses of descriptive data in policy 
development . They are: 

• Use of descriptive data for problem 
definition. Since policy is formed 
in response to problems (either 
actual or imagined), one policy- 
relevant function of descriptive data 
is to quantify phenomena 
objectively so that judgments can 
be made as to whether, and to what 
extent, a problem exists. In this 
respect, SASS data have been 



particularly useful in defining (i.e. , 
quantifying) potential problems 
requiring policy intervention, such 
as teacher qualifications, turnover, 
and shortage (Bobbitt & McMillen, 
1995; Bobbitt, Leich, Whitener, & 
Lynch, 1994). For example, SASS 
data have shown that the percentage 
of teaching positions that are not 
filled is less than 0.5 percent 
(Choy, Henke, Alt, Medrich, & 
Bobbitt, 1993). Thus, there does 
not appear to be a serious shortage 
of individuals available to be 
appointed as teachers. However, 
SASS data have also shown that 
about 6 percent of teachers overall 
do not hold full certification in 
their main teaching assignment. 

This can be taken as evidence of a 
significant shortage of qualified 
teachers. This problem can be 
defined even more precisely by 
computing this shortage percentage 
by main teaching assignment. 

Use of descriptive data for evaluation 
of policies and programs . Without 
collecting evaluation research data 
specifically, descriptive statistical 
data might be used by agencies 
other than NCES (e.g., by policy 
analysts located at various 
governmental agencies and private 
organizations at all levels) to 
evaluate policies and programs. 

For example, descriptive data 
routinely collected on the authority 
vested in school councils (where 
such exist) could be used to assess 
the extent of implementation of a 
state-wide policy to create school 
councils with authority to make 
teacher hiring decisions. 5 In 



addition, SASS could collect data 
about the judgments of principals 
and teachers on the workability, 
utility, and acceptability of policies 
such as those devolving teacher 
hiring decisions to the school level. 
Data might also be collected on 
teacher behavior. For example, if 
it is decided to monitor a policy 
requiring a school improvement 
plan every other year, data might 
be collected from teachers about 
their participation in, and 
contributions to, such planning. 

Data thus collected about the 
judgments and behavior of 
principals and teachers might 
provide insights into why some 
policies seem to be implemented 
and produce desired effects, and 
why others appear not to. Finally, 
some descriptive data relevant to 
school performance (e.g., school 
attendance data, as already 
collected by SASS’s school 
questionnaires) are relevant to 
evaluating policy effects— though, as 
previously observed, most measures 
of school performance must come 
from sources other than SASS. 

Use of descriptive data for 
enlightenment. Apart from having 
immediate and specific relevance to 
defining policy problems and to 
evaluating policy implementation, 
SASS has also collected much new 
background data about schools and 
staffing that is useful to 
policymakers and others in 
fostering a broad understanding of 
the phenomena addressed (i.e, the 
"enlightenment" function of 
research and statistical information 
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as described by Weiss, 1977). 
According to Shavelson (1988), the 
central value of educational 
research findings (and presumably 
the systematic collection of 
statistics) lies in their "... 
constructing, challenging, or 
changing the way policymakers and 
practitioners think about problems" 
(pg. 4). For example, research on 
teacher turnover based on SASS 
data has contributed to a better 
general understanding of the 
components, complexities, and 
magnitude of this phenomenon 
(Ingersoll, Han, & Bobbitt, 1995). 

SASS’s current design emphasizes the 
problem definition and enlightenment uses 
of data collected, and minimizes SASS’s 
relevance to evaluation of policies and 
programs. While the problem definition 
and enlightenment functions of SASS data 
should be continued, it is recommended 
that NCES, in the redesigned SASS, 
attempt to collect more data relevant to the 
evaluation function as well. 

Dimensions of Schooling Amenable 
to Policy Intervention 

The identification of fundamental 
dimensions of schooling that are likely to 
become the subject of policy interventions 
is understood here to be equivalent to 
NCES’s concept of "emerging educational 
issues." In commissioning this paper, 
NCES asked, "What emerging educational 
issues are likely to be important in the 
coming years and how can SASS data 
inform our understanding of these issues?" 
This section of the paper is devoted to 
identifying both "emerging issues" and 



"enduring issues" that are amenable to 
policy intervention. Enduring issues, as 
well as emerging issues, will be considered 
because SASS may not have addressed (or 
sufficiently addressed) either type. The 
second part of NCES’s question, 
pertaining to "how can SASS data inform 
our understanding," will be addressed in a 
subsequent section. 

As addressed in this paper, the 
fundamental dimensions of schooling are 
classified into five main categories: school 
governance, instruction, educational 
finance, school infrastructure, and school 
staffing (principals and teachers). Because 
the first three administrations of SASS 
emphasized school staffing, a subsequent 
section of this paper is devoted to 
consideration of this major topic. 

Therefore, this section of the paper 
concentrates on the other four categories 
(school governance, instruction, 
educational finance, and school 
infrastructure). 

Fundamental dimensions of schooling vary 
in the degree to which they are amenable 
to policy intervention. For example, the 
teacher-pupil ratio is directly amenable to 
policy intervention, while the social 
character of teacher-pupil interactions is 
not, even though the latter may have much 
greater effect on academic achievement 
and other valued student performances. 
Since a primary consideration in the 
redesign of SASS is to inform broad 
education policy pertaining to schooling in 
the next two decades, the objective of this 
section is to identify those fundamental 
dimensions of schooling that have a 
reasonable prospect of being taken 
seriously in future policy formation. Such 
dimensions might be relevant to both 
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current policies (possibly under 
reconsideration for modification or 
discontinuation) and to the formation of 
new policy. 

To identify fundamental dimensions of 
schooling that are likely to be targets of 
policy formation in the next two decades, 
some explicit criteria are required. Such 
dimensions were identified here by 
meeting any one of the following criteria: 

• Dimensions of schooling that have 
been the target of major policy 
formation in the past, especially the 
past ten years or so. 

• Dimensions of schooling that have 
been the target of limited policy 
formation, but have been the target 
of major policy attention (as 
indicated by discussion, debate, 
attempted but failed legislative 
efforts to adopt policies, and 
policies subject to adoption on a 
small scale). 

• Dimensions of schooling that are 
currently of major concern or 
contention to the public and/or 
policymakers. 

A listing of dimensions deemed to be 
fundamental to schooling because they met 
one or more of the above criteria is 
presented in Table l. 6 While other 
observers may offer different lists, there 
probably would be a high degree of 
overlap with the dimensions included here. 



Because the listing in Table 1 includes too 
many dimensions of schooling for practical 
inclusion in redesigned SASS 
questionnaires, a subset of these 
dimensions must be selected. To reduce 
this list, dimensions were selected that met 
all three of the following criteria: 

• Dimensions of schooling that are 
expected to inform broad education 
policy (an NCES specification for 
this commissioned paper). 

• Dimensions of schooling that can 
be measured feasibly by the sample 
survey method with strict limits on 
burden for respondents (e.g., it is 
not feasible to expect LEAs to 
report per pupil costs disaggregated 
by school functions and programs 
because of technical difficulties and 
burden). 

• Dimensions of schooling that are 
not included in other high-quality 
surveys (e.g., the Common Core of 
Data includes LEA financial data). 

A listing of the subset of the dimensions 
meeting each of these criteria is presented 
in Table 2. 7 Other dimensions pertaining 
to teaching force are deferred to the next 
section. Since each of the dimensions of 
schooling selected for Table 2 is a 
candidate for inclusion in the redesigned 
SASS questionnaires for 1998-99, the 
potential of each dimension to inform 
broad education policy in the next two 
decades will be discussed in turn. 
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Table 1: Fundamental Dimensions of Public Schooling Receiving Policy Attention and/or Action*** 



1. School Governance/Organization 

•Centralization/decentralization of authority 

•Regulation/deregulation 

•Accountability 

Student outcomes, and public reporting 
Competition (i.e., school choice) 

Inter- and intradistrict choice 
Charter schools 

Vouchers (including private schools) 
Performance based accreditation 
School performance monetary rewards 

•Privatization 

•Desegregation 

•School safety and discipline 

2. Instruction 

•Curriculum frameworks/standards 

•Opportunity-to-learn standards 

•Student performance standards 

•Assessment of student performance 
Standardized testing 
Performance/authentic measurement 
Attendance 
Completions/drop outs 
Discipline 

•High school graduation standards 

•Instructional practices 

Basic skills vs. higher order thinking 
Course requirements 
Class size 



2. Instruction (continued) 

•Instructional practices (continued) 

Time (hours and days)/scheduling 
Tracking 

Nongraded primary levels 
Inclusion of special need students 

•Instructional materials (mostly textbooks) 

•Instructional technology (computers) 

•Special programs 

Disadvantaged/at risk 

Limited English proficiency 

Special education 

Community service 

Coordinated education, health, 
and social services 

Substance abuse prevention 

School-to-work transition 

•Nontraditional public schools 
Magnet schools 
Vocational education schools 
Schools-within-a-school 
Alternative schools for special 
populations 

Exceptional/nonconforming students 
Special needs students 

3. Educational Finance 

•Sources of school funding 

•Per pupil cost as distributed among 
school functions/programs 

4. School Infrastructure 

•Building construction/rehabilitation 
•Classrooms wired for computers/Internet 
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Table 2: Fundamental Dimensions of Public Schooling Recommended for Inclusion in SASS, 
along with Estimations of Costs of Implementing Related Education Policies 



1 . School Governance 

{Centralization/decentralization of authority} ** 
Regulation/deregulation * N 
Privatization * R 

2. School Accountability 

School outcomes ** 

{Public reporting of school outcomes} * N 
Competition (i.e., school choice) * R 
{Inter- and intradistrict choice} 

Charter schools 

Vouchers (for public and private schools) 

3. Standards: Curriculum and Student Performance 

Curriculum frameworks/standards ** 

Student performance standards ** 

Associated with curriculum standards 
{High School graduation standards} 

4. Assessment of Student Performance 

Standardized testing ** 

Performance/authentic measurement ** 
{Attendance} * N 
{Completions/drop outs} * N 

5. Instructional Practices 

Basic skills vs. higher order thinking ** 
Inclusion of special need students * R 

6. Instructional Technology 

Computer usage and infrastructure *** 

7. Special Programs 

{Disadvantaged/at risk} * * * 

{Limited English proficiency} *** 

{Substance abuse prevention} ** 

8. Nontraditional Public Schools 

{Magnet schools} ** 

{Alternative schools for special populations} ** 
{Schools-within-a-school} * * 



* N Policy implementation would entail only minor incremental funding by new appropriations. 
* R Policy implementation would entail reallocation of existing funding, but little new funding. 

** Policy implementation would require substantial incremental funding. 

*** Policy implementation would require major incremental funding. 

{ } Dimensions of schooling addressed by the 1993^94 SASS. 



School Governance 

School-based management. In recent years, 
a great deal of policy development in 
education has involved changes in school 
governance, most of which has focused on 
school-based management— a policy 
designed to transfer authority from the 
LEA to the school level. Since major 
policy interest and development on the 
locus of authority over various aspects of 
schooling are expected to continue in the 
future, SASS data could be very useful in 
tracking the status of, and changes in, 
authority vested in school boards, 
superintendents, school councils, principals 
and teachers. 

Deregulation. Deregulation is another 
school governance topic that has been the 
subject of much policy discussion and 
formation over the past decade or so, at 
both the federal and state levels. 
Deregulation typically is intended to 
liberate school personnel from stifling 
regulations so that they will be able to 
change school programs and practices in 
ways judged to be most responsive to local 
needs and circumstances. Deregulation 
policies have been framed in several ways, 
such as waivers of regulations granted 
upon application submitted by schools, 
deregulation for high performing schools, 
revocation of regulations by the regulating 
authority, and the award of charter school 
contracts with much less regulation than 
applicable to regular public schools. Since 
major policy interest and development in 
the area of school regulations is expected 
to continue in the future, it is important to 
track this phenomenon with SASS data. 

Privatization. Privatization of instruction 
and school management is the third school 



governance topic of importance for 
monitoring by SASS data. Privatization is 
the subject of rapidly increasing policy 
interest development, especially during the 
past year when the Republican party 
captured majorities in Congress and in 
most state legislatures. The two main 
forms of privatization at the present time 
are: contracting for the management of 
particular schools, and contracting with 
private organizations to operate and staff 
charter schools— intentionally designed to 
have full authority and little regulation. 
Some vocal policy analysts and 
policymakers further advocate that school 
improvement can only be gained through 
radical change in the functions of LEAs, 
namely, that LEAs should function only as 
policy bodies and as contracting agencies 
for public schooling. Under this 
conception, all "public" schools would be 
operated by private corporations under 
contract with LEAs, and be subject to 
LEA policy and monitoring for contract 
compliance. 

Privatization, in its various forms, is an 
appealing option to policymakers for 
several reasons, one of which is that it 
requires little or no incremental funding. 
Instead, the costs of privatizing schools are 
largely underwritten by reallocation of 
existing funding for regular public schools. 
In view of the rapidly increasing interest in 
privatization of schooling, the collection of 
SASS data about this phenomenon could 
be very helpful in future policy 
development. 

School Accountability 

School accountability measures, programs, 
and systems have been the subject of much 
policy development during the past decade 
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or so, and this is almost certain to be an 
area of much policy action in the coming 
decades. Though accountability policies 
have also been adopted for LEAs, school 
administrators, teachers, and students, the 
focus here is on accountability policies 
applicable specifically to public schools. 

Measuring school activities. One 
accountability strategy favored by 
policymakers is the measurement of school 
outcomes, especially by standardized 
achievement tests and various approaches 
to performance measurement, and also 
drop out percentages. Achievement testing 
occurs at the national level (i.e., the 
National Assessment of Educational 
Progress— NAEP) and in most states. 

SASS could be very useful in collecting 
data on (a) the types and extent of outcome 
measurements of all kinds administered in 
schools, (b) the time and effort the mea- 
suring operations consume (including 
preparing students for testing), (c) the 
impact such measures have on instruction 
(e.g., teaching-to-the-test), and (d) the 
views of school personnel on the utility of 
various types of outcome measures for 
improving school performance. 

Public reporting of school outcomes. Once 
states are committed to measuring school 
outcome performances, the public 
dissemination of such measures is a widely 
used accountability policy of very low 
incremental cost. It is popular with the 
press and the public who are very interest- 
ed in school rankings, and this kind of 
exposure brings public pressure on princi- 
pals and teachers to explain performance 
levels attained, and to develop and report 
school improvement plans. Because public 
reporting of school performance is so 
popular and costs so little, it is expected to 



continue to be widely mandated. SASS 
could collect useful data on the impact of 
this policy on the attitudes and behavior of 
school personnel. Such data would be 
helpful in assessing whether this account- 
ability policy has the intended effect of 
evoking school improvement efforts. 

Promoting competition. Other 
accountability policies are designed to 
promote competition among public 
schools, and between public and private 
schools. These policies entail various 
schemes for intra- and interdistrict public 
school choice, charter schools, and 
vouchers. School competition has been an 
area of increasing policy ferment in recent 
years, is so at present, and almost 
certainly will be so in the future. In fact, 
charter school and voucher polices are 
perhaps the subject of the most intense 
policy debates at the present time. SASS 
can provide useful data for informing 
broad education policy by tracking the 
several facets of this phenomenon in terms 
of how it impacts on the functioning of 
schools, on the work of their staffs, and on 
school climate. 

The competitive aspects of school choice, 
charter schools, and vouchers have been 
discussed here as accountability strategies 
used by policymakers. It should be recog- 
nized, however, that these strategies serve 
other purposes as well. One of these 
purposes is to provide alternatives for 
students whose particular needs are not 
being served well by instruction and other 
programs offered in regular classrooms. 
Some advocate that such students should 
be able to choose a type of school best 
suited to them. Hence, a variety of school 
choices is required for this purpose. In 
addition to competition, privatization of 
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schooling serves a further function. As 
noted above under school governance, the 
charter school and voucher varieties of 
school competition are also strategies used 
to remove schools from the direct control 
of LEA’s and to place them under private 
auspices. 

Standards: Curriculum and Student 
Performance 

Curriculum. The development of 
curriculum standards represents a major 
current policy initiative at the federal, 
state, and local levels, even though there 
has been considerable conflict over 
standards developed in some subject 
matters. Because the development of 
standards represents a major policy trend 
that lies at the center of the teaching and 
learning process (as distinguished from, 
for example, school governance), it would 
be worthwhile to track the types and 
sources of curriculum standards used in 
the nation’s schools. 

Student performance . Even though the 
development of student performance 
standards trails behind the development of 
curriculum standards, it would also be 
worthwhile to track whether student 
performance standards are used, and, if 
so, their source and whether they are 
correlated with curriculum standards that 
may be in use. 

Opportunity-to-leam standards are omitted 
here because NAEP surveys already 
address instructional practices in some 
detail. 

High school graduation standards are 
recommended for inclusion in SASS 
because they have also been the target of 



policy attention in recent years. They 
represent the level of attainment expected 
by the time of completion of secondary 
school, and are relevant to school-to-work 
and school-to-college transitions. 

Assessment of Student Performance 

The assessment of actual student 
performance (as distinguished from 
expected standards of performance) is a 
major ongoing subject of policy formation 
and is central both to instruction and to 
school performance. SASS can have two 
important roles in tracking the assessment 
of student performance. One role is to 
survey the types of measures used for 
academic achievement, especially 
standardized and performance/authentic 
measurement. However, it is not expected 
that SASS will attempt to collect data on 
the results of such measures due to the 
burden involved and because these results 
are often available from state education 
agencies. The second role for SASS is to 
collect data on the level of student 
performance based on other types of 
indicators (e.g., attendance data) because 
these data are important and the burden is 
modest. 

Instructional Practices 

As noted above, NAEP surveys address 
instructional practices in some detail. 
Therefore, little in this category is 
recommended for SASS coverage except 
for two instructional practices that are not 
covered by NAEP, have been contentious 
with educators and parents, and have been 
the subject of continuing policy attention. 
The first of these two practices is the 
instructional emphasis placed on basic 
skills in contrast with the emphasis on 
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higher order thinking. The second 
instructional practice is the inclusion of 
"special needs" students in regular 
classrooms. If SASS data are collected on 
these two practices, it would also be useful 
to survey the type and amount of 
instructional emphasis given to four other 
topics that have been of interest in the 
development of policy, namely, discipline, 
working together cooperatively, values, 
and computer literacy. 

Instructional Technology 

Many policymakers and others expect 
microcomputers in the classroom to 
revolutionize instruction, and continuing 
policy attention to this topic is expected. 
SASS can inform policy development in 
this area by collecting data about the 
availability in classrooms of computer 
hardware, networking and access to the 
Internet, and software for instructional 
purposes, how and the extent to which it is 
used, and other important aspects of this 
technology (e.g., the availability in schools 
of skilled technicians to install and 
maintain this technology). 

Special Programs 

A number of special programs have been 
implemented in public schools to address a 
variety of social problems (especially those 
of poverty, limited English proficiency, 
and substance abuse) that limit student 
performance. These concerns are expected 
to continue to command the attention of 
policymakers, educators, and the public in 
coming decades— especially since current 
trends suggest that such social problems 
are intensifying. SASS can contribute to 
policy development in this area by 
collecting data about the extent of such 



problems in schools, and the types and 
extent of school programs designed to 
address them. 

Nontraditional Public Schools 

Nontraditional schools (e.g., magnet 
schools, schools-within-schools, alternative 
schools, and specialized schools for 
students with severe disabilities) have been 
established in response to a variety of 
social and human concerns such as 
desegregation, school drop outs, 
alienation, and learning and physical 
handicaps. Since the concerns to which 
these schools respond will continue in the 
decades to come (and may even intensify), 
SASS data would be useful in informing 
policy review and development in this 
area. 

The strategy to break up very large (and, 
therefore, impersonal) schools into smaller 
schools (i.e., schools-within-a-school) is an 
instance of a larger strategy designed to 
improve schools. Data that are collected 
routinely by SASS on school size and type 
are useful for tracking changes in these 
dimensions. 

Review of Prior SASS 
Questionnaires 

Improvements in, and changes to, the 
contents of SASS questionnaires have been 
made by NCES for the 1990-91 and 1993- 
94 administrations. There was substantial 
deletion of content from the 1990-91 
survey and addition of new content for the 
1993-94 survey. Many of the items 
deleted from the 1990-91 survey pertained 
to teacher supply and demand. Some of 
these changes also involved the deletion of 



items from the school questionnaires that 
duplicated similar items from the teacher 
questionnaires. Nonetheless, in 
commissioning this paper, NCES asked, 
"Is the existing balance between supply 
and demand issues and other school topics 
still appropriate?" The review of SASS 
questionnaires presented here will yield an 
answer to this question about priorities. 

Argument for improving SASS. As in the 
past, NCES can easily justify making 
incremental improvements in, and changes 
to, SASS. That is not the issue. The 
issue is whether the content of SASS 
questionnaires for 1998-99 should 
represent a drastic change from the past— a 
change characterized by major reductions 
in content pertaining primarily to teacher 
supply and demand (but also to principal 
supply and demand), and the addition of 
new content about other dimensions of 
schooling that would better inform broad 
education policy. 

The argument for changing SASS 
drastically is that the first three rounds 
have yielded substantial data about the 
teaching force and additional data are 
unnecessary. Also, that by continuing to 
emphasize teacher variables, an 
opportunity is lost to collect data about 
aspects of schooling that will be of 
importance to future policy development 
on a wide range of other issues. 

Argument for not modifying SASS. On the 
other hand, an argument can be made to 



maintain SASS in its 1993-94 form. Past 
decisions to emphasize collection of data 
about teachers and principals were astute 
and justified because staffing is the central 
factor in determining the quality and 
improvement of schooling. This view is 
buttressed by the results of a recent large- 
scale meta-analysis of education production 
functions that found that "resource 
variables that attempt to describe the 
quality of the teachers (teacher ability, 
teacher education, and teacher experience) 
show very strong relations with student 
achievement" (Laine, Greenwald, & Hedg- 
es, 1995, pp. 57-58). It can further be 
argued that, since the three past SASS 
administrations have yielded a large 
amount of data basic to understanding the 
dynamics of the teaching force (including 
trends over time), it is vital that continuity 
in data collection be maintained about this 
most important component of the quality 
and character of schooling. 

Recommended approach. In response to the 
genuine tension between the strategies of 
"drastic change versus continuity" in 
redesigning SASS for 1998-99, this paper 
takes a middle position. While a consider- 
able amount of the content of the 1993-94 
SASS questionnaires can be compressed or 
deleted to accommodate expansion in other 
content, the first priority is to maintain 
continuity in data collection basic to 
understanding the attributes and flows of 
the teaching force. The basic data 
collection to be maintained for this 
purpose is presented in Table 3. 
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Table 3: Fundamental Dimensions of the Teaching Force Addressed by SASS Questionnaires 



1 . Teacher demand 

2. Sources of supply 

3. Teacher shortage 

4. Teacher demographic characteristics 

5. Teacher qualifications 

•Teacher preparation 
Preservice 

Professional development 
•Certifications 
• Experience 

6. Teaching assignment and load 

7. Employment and working conditions 

8. Collective bargaining/union membership 



9. Turnover 

•Among schools 
•Among teaching fields 
•Between sectors 
•Attrition 

10. Compensation 

•Level of compensation 
•Minimum compensation 
•Special monetary incentives 
•Salary setting principles 
Salary schedules 
Merit pay 
Career ladder 
•Employee benefits 



Although it is widely recognized that 
teacher quality is perhaps the most critical 
educational determinant of student 
achievement (Kennedy, 1992; Mandel, 
1995; Laine, Greenwald, & Hedges, 

1995), teacher quality is a broad concept 
that includes (a) teacher qualifications, (b) 
classroom teaching performance, and (c) 
teacher ability such as measured by tests. 
The sample survey method is very useful 
for collecting data on teacher 
qualifications, and SASS should continue 
to have a strong emphasis on collecting 
such data as it has in the past. SASS is 
not a suitable vehicle for collecting data on 
the quality of classroom teaching 
performance, even under simulated 



conditions. Although teaching 
performance is the prime facet of teacher 
quality, a promising research approach to 
investigating this topic (as distinguished 
from collecting survey data) has been 
recommended elsewhere (Mandel, 1995). 

As to teacher tested ability, NCES should 
study (if it has not already done so) the 
feasibility of either (a) collecting tested 
ability scores for teachers in the SASS 
sample, or (b) linking SASS to other 
databases where such scores might be 
recorded. Even tested ability scores for a 
subsample of the SASS sample of teachers 
would be useful. 



Teacher qualifications are also directly 
relevant to the dimension of teacher 
shortage, as listed in Table 3. Information 
derived from SASS data demonstrates that 
there is practically no shortage of the 
numbers of individuals that are willing and 
able to accept teaching positions (Choy et 
al., 1993). Instead, other research has 
shown that there is a shortage of qualified 
teachers, and an even greater shortage of 
high-quality teachers (Boe & Gilford, 

1992; Gilford & Tenenbaum, 1990). 
Therefore, the continued collection of 
extensive data on teacher qualifications by 
SASS will be vital to measuring the level 
of, and trends in, the shortage of qualified 
teachers. 

As in past administrations of SASS, it is 
recommended that data pertaining to 
dimensions of the teaching force listed in 
Table 3 should continue to be collected 
through the teacher demand and shortage, 
school, and teacher questionnaires. 
However, the breadth of coverage of these 
dimensions should be reduced to a 
reasonable minimum. Some specific 
suggestions for minimizing the breadth of 
coverage of some of these dimensions are 
made below. One general 
recommendation is to eliminate all 
redundant item content that may still exist 
between questionnaires and within 
questionnaires. 8 

With respect to the remaining content of 
SASS questionnaires for 1993-94 (i.e., 
other than content relevant to the 
dimensions of the teaching force identified 
in Table 3), much should be condensed or 
deleted to accommodate new content in 
1998-99 that will better inform broad 
education policy. The following guidelines 



will be helpful in deciding what content to 

retain, condense, or delete: 

• Content basic to understanding the 
characteristics, qualifications, 
responsibilities and authority, 
leadership style, and compensation 
of school principals should be 
retained, but be reduced in depth of 
coverage. 

• Content should be retained that has 
the potential to inform broad 
education policy in the coming two 
decades, while content that does not 
should be minimized. The topics 
listed in Tables 2 and 3 suggest 
what this content should be. 

• Content should be minimized that 
is, or will be, available from other 
high-quality surveys. 

• Content should be minimized that 
has shown a stable pattern over the 
first three SASS administrations if 
it is likely to remain stable or can 
be predicted with reasonable 
accuracy. 

• Content should be minimized that 
has generated little policy or 
research interest in the past, unless 
there is reason to believe that it 
will become useful to emerging 
policy issues. 

• Content should be minimized that is 
based on questionnaire items of 
marginal technical quality (unless 
efforts to improve the items are 
successful). 
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Content of Main SASS Questionnaires 

Based on the general considerations 
discussed above for redesigning SASS 
questionnaires, specific observations and 
suggestions are made below relating to the 
content of each of the four main SASS 
questionnaires used in 1993-94. 

Teacher demand and shortage questionnaire. 
The Teacher Demand and Shortage 
Questionnaire is the only source of some 
teacher data and should continue to be 
collected from LEAs. Items such as 
district teacher counts, hiring criteria, and 
collective bargaining agreements should be 
retained. In contrast, items pertaining to 
teacher type, certification, supply, 
turnover, ethnicity, and retirement benefits 
are candidates for deletion, because such 
information is included in the Public 
School Teacher Questionnaire. Much of 
the other content of the Teacher Demand 
and Shortage Questionnaire addresses the 
dimensions of schooling listed in Table 2, 
and are, therefore, candidates for 
retention. 

Principal questionnaires. Though collection 
of data on the education and experience of 
principals is important and should 
continue, it would seem that assigning 
two-thirds of the content of the question- 
naire to these two topics is excessive, and, 
therefore, should be condensed. While the 
item pertaining to school problems may 
continue to be important, it is doubtful that 
the same item should continue to be 
included in both the principal and teacher 
questionnaires. The content on locus of 
influence is relevant to governance policies 
promoting decentralization. Collection of 
data on this general topic should continue, 



but consideration might be given to casting 
it in terms of decision-making authority. 

School questionnaires. Much of the content 
devoted to the teaching force in the first 
two administrations of the school 
questionnaires was eliminated for the third 
administration in 1993-94. As last 
designed, the content of the school 
questionnaires was focused on basic 
descriptive information about school 
characteristics, staffing, programs, and 
policies. Much of this is essential to 
describing schools, and much is relevant to 
educational policy issues. Therefore, it 
would probably not be prudent to delete or 
compress much of this material. 

Teacher questionnaires. The content of the 
teacher questionnaires needs to be trimmed 
to permit the inclusion of expanded content 
relevant to education policies in areas 
other than the teaching force. Some 
suggestions are: (a) compress the content 
on teacher experience (especially breaks in 
service and experience prior to beginning 
teaching), (b) eliminate content on changes 
in teaching assignment (as data on year-to- 
year changes are available from the TFS), 
(c) compress the content on teaching load, 
and (d) compress or eliminate much of the 
content on teacher perceptions and 
attitudes (depending upon the extent to 
which previous analyses have demonstrated 
stability in data pertaining to these topics, 
and the extent to which these data have 
proven to be interesting or useful to 
policymakers and others). While these and 
other changes might be made to data 
collected from teachers, it is important to 
continue to collect sufficient data to 
monitor all the dimensions of the teaching 
force listed in Table 3. 



Teacher Followup Survey 



As a longitudinal component of SASS, the 
TFS has served an indispensable role in 
monitoring year-to-year flows of teachers 
included in the prior SASS sample. As a 
vehicle for tracking actual teacher career 
transitions (as distinguished from teacher 
reports of activities in the prior year, and 
plans for the coming year), TFS is the 
definitive means for collecting data on 
teacher turnover and variables associated 
with teacher turnover. Therefore, TFS 
should be retained in much the same form 
as in the past. 

Linking SASS with Student Data 

Because data collected by SASS provides 
critical national- and state-level informa- 
tion about schools and their staffing that is 
unavailable from others sources, SASS 
most definitely should be continued in its 
present general form. Yet, a major 
limitation of SASS is that it does not 
include student achievement data that can 
be analyzed in relation to school, teacher, 
and principal variables. Fortunately, 

NCES is exploring the possibility of 
linking SASS to student data collected by 
other surveys such as NAEP and the 
National Eduction Longitudinal Survey of 
1988 (NELS:88). The advantages of 
doing so are obvious if the important role 
served by SASS in NCES’s current array 
of surveys is not greatly diluted or 
sacrificed. If such radical changes in 
SASS were required to link it with student 
data that much of its current value to the 
field would be lost, then other solutions 
should be sought (e.g., expanding or 
otherwise changing the teacher and 
school/school administrator questionnaires 
of NAEP and NELS:88). 



Discussion and Recommendations 

Feasibility of Expanding SASS Content 

To recapitulate, it is recommended that 

future data collection by SASS be 

prioritized as follows: 

1. Include fundamental dimensions of 

the teaching force as listed in Table 
3, with emphasis on teacher 
qualifications. 

2. Include basic attributes of school 

principals, LEAs, and schools. 

3. Include fundamental dimensions of 

schooling that are expected to be 
active areas of policy development 
in the next two decades, as listed in 
Table 2. 



This is to be accomplished without 
increasing the burden on SASS 
respondents. 

In contemplating the feasibility of adopting 
these recommendations, it should be 
recognized that the four SASS 
questionnaires used in 1993-94 already 
addressed a substantial majority of the 
dimensions recommended here. The only 
new topics recommended for inclusion in 
the next SASS are some of the dimensions 
of schooling listed in Table 2. As can be 
seen in this table, the 1993-94 SASS 
collected data pertaining to half of the 
dimensions listed. SASS has also collected 
data on all the teacher, principal, and 
school variables recommended above for 
continued coverage in the 1998-99 SASS. 
To offset the burden created by adding 
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new content, other recommendations were 
made about compressing or deleting 
content from the four SASS questionnaires 
used in 1993-94 (see section V). 

It, therefore, seems feasible to consider 
redesigning SASS in accordance with the 
recommendations offered in this paper. 

The fact, however, that past SASS 
questionnaire content included half the 
dimensions of schooling listed in Table 2 
does not imply that their coverage was 
adequate (either in form or breadth) for 
future purposes. Therefore, it may be 
necessary to establish further priorities for 
selecting among the specific dimensions of 
schooling recommended in Table 2 for 
future data collection, as addressed in the 
next section. 

Data Collection Priorities for Dimensions of 
Schooling 

As noted above, a number of the 
dimensions of schooling listed in Table 2 
represent new content areas recommended 
for data collection by SASS because of 
their potential relevance to future policy 
development. If it is not feasible to collect 
SASS data about all these dimensions, then 
the subset selected should include the 
dimensions that are likely to be the most 
active areas of policy debate and 
development. The best candidates for 
areas of most intense policy development 
are school governance and accountability— 
just as they have been in the recent past. 

It is, therefore, worth examining why 
policymakers have attended so extensively 
to aspects of school governance and 
accountability. 

A primary reason for the attention given 
by policymakers to these two facets of 



schooling is related to the "means" 
available to policymakers to influence the 
educational process. Specifically, 
policymakers can influence education by 
the following means. 9 

• They can structure the governance 
context by: 

— Allocating decision-making 
authority (such as in school-based 
management). 

— Adopting or revoking mandates 
(i.e., in the forms of statutes and 
regulations) (such as deregulation). 

— Designating public or private 
sector for operations (such as in 
privatization). 

• They can establish accountability 
policies, with performance 
incentives, designed to: 

— Measure and report school 
performance (such as achievement 
test scores). 

— Promote competition among 
schools (through school choice, 
charter schools, vouchers). 

— Link rewards and sanctions to 
school performance (such as 
offering monetary rewards). 

® They can appropriate funds by: 

— Making direct allocation of funds 
for programs, and to build capacity 
in terms of human, equipment, or 
infrastructure resources (such as 
computer acquisitions). 
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— Creating financial incentives to 
evoke desired responses (such as 
school improvements). 

Adopting governance and accountability 
policies is appealing to policymakers for 
several reasons: (a) the funding required is 
relatively low, (b) implementation can 
often be accomplished quickly by central 
action without the delay and difficulty 
involved in securing the participation of 
professional educators (i.e., district 
superintendents, principals, and teachers), 
(c) policymakers appear to be decisive and 
to have educational problems under 
control, and (d) the policies often enjoy 
broad public support. For all these 
reasons, implementation of governance and 
accountability policies represents good 
politics. Therefore, such policies have 
been quite popular with policymakers for 
at least the past decade, and probably will 
continue to be so in the foreseeable future. 
In the following paragraphs, the three 
means available to policymakers to 
influence the educational process are 
discussed in greater detail. 

Through structuring the governance 
context of schools, policymakers can 
control the governance environment in 
which schools operate by (a) assigning 
decision-making authority along the 
continuum from centralization to 
decentralization (the main levels of the 
continuum being Federal, state, LEA, and 
school), (b) adopting or revoking statutes 
and regulations (both as to the content and 
the general extent of control of schooling), 
and (c) designating the functions of 
schooling that will be performed by 
agencies in the public and private sectors 
(such as privatizing the management of 
public schools). As indicated in Table 2, 



only minor or moderate incremental 
funding is required to implement policies 
in the governance areas. Furthermore, 
policy decisions about school governance 
can often be translated into action much 
more quickly than can changes in 
instruction. Consider, for example, the 
relative ease with which regulations can be 
repealed in comparison with implementing 
curriculum standards in the classroom. 
These characteristics of school governance 
policies help explain why they have been 
so prevalent in recent years, and why they 
are expected to be a major area of policy 
development in the coming decade or two. 
A high priority should be placed by NCES 
on collecting SASS data about school 
governance arrangements in the interest of 
informing broad education policy. 

Policymakers also influence schooling by 
establishing accountability procedures and 
systems applicable to LEAs, schools, 
teachers, administrators, and students. 

Such policies are popular with the press 
and with the public because they are taken 
as evidence of engagement, oversight, and 
control by responsible authorities. As 
indicated in Table 2, a remarkable variety 
of school accountability strategies are 
available to policymakers and many of 
these have been used widely (Boe, Boruch, 
Landau, & Richardson, 1993), while only 
minor or moderate incremental funding is 
required to implement them. In addition, 
such policies can often be implemented 
quickly and easily since they do not 
require the collaboration of school 
personnel. These circumstances explain 
why the development of accountability 
policies has been so prevalent in the past 
decade or so, and why it is reasonable to 
expect that this will continue to be a very 
active area of policy development in the 
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future. It is, therefore, recommended that 
NCES place high priority on collecting 
data about various forms of school 
accountability. 

The appropriation of funds to improve 
schooling is, of course, a means also used 
extensively by policymakers. As noted 
above, one of the main advantages of 
developing policies of the school 
governance and accountability types is that 
they generally do not require substantial 
funds for implementation. However, the 
appropriation of substantial or 
major funds is usually required to 
implement policies intended to improve 
instruction such as curriculum and 
performance standards, educational 
technology, and special programs such as 
for at-risk students (see items 3 through 8 
of Table 2). At this time of retrenchment 
in government spending, tight finances 
alone represent a major impediment to 
rapid progress in improving instruction. 

With respect to adopting policies intended 
to improve instruction, several factors 
other than funding limitations also pose 
serious difficulties to implementing such 
policies. For example, efforts to 
implement curriculum, opportunity-to- 
leam, and performance standards have 
become embroiled in controversy over the 
proper role of federal and state policy in 
this area. Policies addressing other 
instructional issues (such as authentic 
assessment, inclusion of special needs 
students in regular classrooms, programs 
for limited English proficient students, and 
so on) are also very controversial. In 
addition, implementation of policies 
intended to improve instruction usually 
involve (a) changes in the work of 
principals, teachers, and students, and (b) 



changes in school programs and practices— 
both time-consuming processes. For all 
these reasons, progress in improving 
instruction is likely to be slow and uneven- 
-even though central to improving teaching 
and learning. Regardless, it can be 
expected that policymakers will continue to 
devote considerable attention to 
instructional issues in the coming decades, 
and SASS data can be very helpful in 
informing broad education policy in this 
area as well. 

The considerations discussed above suggest 
that policy development in the areas of 
school governance and accountability will 
be particularly active (and implementation 
of such policies will be feasible) during the 
next two decades, while policy 
development in the area of instruction will 
continue to be fraught with great 
difficulty. Therefore, if priorities need to 
be established for SASS data collection 
among these three areas, it is 
recommended that priority should be 
accorded to school governance and 
accountability in the interests of informing 
policy deliberations. 

Evaluation Function of SASS Data 

To enhance the value of SASS data for 
policy development, it has been 
recommended above that SASS 
questionnaires be designed to collect more 
data that is useful in evaluating policies, 
programs, and practices. While NCES 
should maintain a neutral posture with 
respect to the import of such data to policy 
issues, it seems feasible for NCES to 
collect and report descriptive statistical 
data of this type. In fact, such data were 
collected in the 1993-94 SASS, as 
illustrated by an item in the teacher 
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questionnaires which sought teachers’ 
opinions about the impact and value of 
professional development programs in 
which they had recently participated. 
However, an item of this type represents 
only one of four types of descriptive data 
that could be very useful for assessing 
policies and programs. The several types 
of data relevant to evaluation that might be 
collected through SASS are: 

1 . Data on policies adopted. Data on 
whether an LEA has adopted a 
policy locally can be useful in 
evaluating the acceptability or 
feasibility of a policy originating at 
a higher level. For example, a 
state may promote (say, through 
financial inducements and provision 
of technical assistance) the 
voluntary adoption of school-based 
management by LEAs. The 
incidence of adoption of the new 
policy would be relevant to 
assessing the strategy used by the 
state to promote this policy. 

2. Data on policies implemented as 
programs or practices. In those 
instances where an LEA has 
adopted a policy, data on whether it 
has been implemented at the school 
level in the form of a program or 
practice also can be useful in 
evaluating the acceptability or 
feasibility of a policy. If a policy 
has not been implemented, further 
data can be collected on barriers to 
policy implementation; if a policy 
has been implemented as a program 
or practice, further data can be 
collected that describe the program 
or practice, and any unexpected 
side effects. Such information can 



be useful for assessing whether the 
program or practice embodies the 
basic intent of the policy. For 
example, information could be 
collected about school-level efforts 
to implement a school-based 
management and the specific form 
taken by this management 
arrangement. 

3 . Data on judgments by principals and 
teachers. In those instances where 
a policy has been implemented at 
the school level, the judgments of 
principals and teachers can be 
collected about the workability, 
utility, and acceptability of the new 
programs or practices that result 
from implementing the policy. For 
example, the judgements of 
principals and teachers on various 
aspects of a school-based 
management system implemented in 
their school could be useful in 
assessing the merits and liabilities 
of delegating operational authority 
to the school level. 

4. Data on behavior of principals and 
teachers. In those instances where 
a policy has been implemented at 
the school level, reports by 
principals and teachers could be 
collected about changes in their 
behaviors that have occurred as a 
result of implementing a policy. 

For example, the work of 
principals is expected to be changed 
substantially by the introduction of 
school-based management. If the 
workload has shifted, for example, 
from 50 percent instructional 
support and 10 percent financial 
management (plus other functions), 



to 10 percent instructional support 
and 50 percent financial manage- 
ment, such information would be 
useful to policymakers and others 
to assess the policy impact on the 
culture of schools. 

The discussion above demonstrates that 
descriptive statistical data such as collected 
by SASS could be very useful in assessing, 
refining, and developing education policy. 
If the evaluation function of SASS is 
construed in this way, it represents an 
approach to securing the systematic (as 
distinguished from anecdotal) input of 
principals and teachers on initiatives that 
have been taken, or might be taken, to 
improve schooling. Policymakers and 
others would then have sound information 
about the views of principals and teachers 
about what is workable and useful, and 
what is not. Therefore, the evaluation 
function of SASS data should be enhanced. 



Summary 

The final question posed by NCES in 
commissioning this paper was: "What are 
the likely concerns of the next 10-20 
years, and what data should we collect 
now to inform those issues in the next 
decade?" The short answer to the first 
part of this question is that SASS should 
collect data relevant to the fundamental 
dimensions of public schooling listed in 
Table 2 and the dimensions of the teaching 
force listed in Table 3, for reasons 
previously discussed. If it is not feasible 
to collect data on all the dimensions listed 
in Table 2, then it is recommended that the 
priorities for expanding SASS 
questionnaire content should include the 
various dimensions of school governance 



and accountability listed in Table 2, rather 
than all the dimensions relating to 
improvement of instruction. 

The guidelines that were used in this paper 
to select the dimensions listed in Tables 2 
and 3 were: 

• Select fundamental dimensions of 

schooling, especially of public 
schools 

• Emphasize the selection of 
dimensions amenable to policy 
intervention at all levels 

• Emphasize the selection of 
dimensions pertaining to the 
teaching force because of the 
central role of teachers in the 
quality and improvement of 
schooling, and in the interest of 
continuity with past SASS 
administrations 

• Emphasize the selection of 
dimensions of schooling that have 
the greatest potential to be policy 
issues during the next two decades 

• Include dimensions of major 
concern to stakeholders 

• Exclude dimensions for which 
nationally representative data are 
collected in other high-quality 
surveys 

• Exclude dimensions that are 
inappropriate for questionnaire 
surveys 

• Exclude dimensions that pose an 
unacceptable burden on respondents 





209 



After thus having selected the sets of 
dimensions listed in Tables 2 and 3 for 
inclusion in SASS questionnaires, the 
second part of NCES’s question can be 
addressed, namely, "What data should we 
collect now to inform those issues in the 
next decade?" In summary, it was 
recommended that: 

• Data should be collected to quantify 
the baseline status of schools and 
their staffs with respect to the 
dimensions selected for inclusion in 
SASS questionnaires 

• Once baseline status is established, 
subsequent administrations of SASS 
should monitor possible changes 
from baseline status in the 
dimensions of interest 

• The collection of data on both 
baseline status and changes from 
baseline should be designed so that 
the data are useful to inform 
education policy development. The 
value of such data for policy 
development will be maximized if 
the data are: 

— Useful for problem definition, 

— Useful for evaluation of policies 
and programs, and 
— Useful for enlightenment 

In the past, SASS data have been 
especially useful for problem definition 
and for enlightenment. In the future, it is 



recommended that questionnaires be 
designed to continue to collect useful data 
for these purposes, and, in addition, be 
designed to collect data more useful for 
evaluation of policies and programs. 
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1. This paper assumes that the reader is familiar with background information about SASS. In brief, 

SASS is a large-scale cross-sectional survey with different questionnaires being administered to 
independent national probability samples of local education agencies (LEAs), schools, principals, and 
teachers in the public sector (variations of the questionnaires for schools, principals, and teachers were 
administered in private and Indian schools). In the public sector, schools are sampled first, and 
teachers are sampled within the schools. In addition, the principals for the sampled schools and the 
LEAs in which the schools are nested are included. Thus, responses to the several questionnaires can 
be linked. During the year following a SASS administration, the Teacher Followup Survey (TFS) is 
administered to three subsamples of teachers as follows: (a) teachers who continued to teach in the 
same school as in the SASS year, (b) teachers who transferred to a different school in year after SASS, 
and (c) teachers who left the teaching profession at the end of the SASS year. Descriptive information 
about SASS and TFS is available from NCES. 

2. Throughout this paper, references to SASS alone imply SASS and TFS. 

3. In fact, baseline status data may often be of more value in informing broad education policy than data 
on changes from baseline. Therefore, when this paper discusses changes in school policy, programs, 
practices, and performance, it should be understood that establishing baseline status data is a necessary 
and integral part of measuring change. 

4. The major examples are school attendance items in the School Questionnaires, and school climate in the 
Principal and Teacher Questionnaires. 

5. The Public School Questionnaire of SASS’s third round already collects information about several 
functions of school councils. 

6. Other than the dimensions of school listed in Table 1, SASS has also collected data on a variety of 
basic aspects of schooling such as student enrollment in LEAs and schools, staffing pattern and size, 
school type and level, location, etc., and should continue to do so. 

7. It is possible that data for some dimensions included in Table 2 are available from other NCES surveys 
and should, therefore, be deleted from this list. This can best be determined by NCES staff members 
who know the detailed content of all their surveys. 

8. It is recognized that most or all of such redundancies were eliminated in the 1993-94 SASS. 

9. The following outline is based in part on the identification of policy instruments by McDonnell and 
Elmore (1987). 
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1998-99 SCHOOLS AND STAFFING SURVEY: 
ISSUES RELATED TO SURVEY DEPTH 

Susan P. Choy, MPR Associates, Inc. 



Introduction 

The Schools and Staffing Survey (SASS), 
first fielded in 1987-88, represented a 
major redesign of the NCES elementary 
and secondary data collection, and it has 
proved to be a successful one. 

Thoughtfully designed and competently 
executed, the SASS linked surveys have 
made it possible to develop comprehensive 
profiles of the nation’s public and private 
schools and teachers and principals, and to 
examine supply and demand issues more 
thoroughly than ever before. Successive 
administrations of SASS in 1990-91 and 
1993-94 brought improvements in sample 
and questionnaire design and supplied 
additional information on timely topics. 
The switch to a five-year cycle of data 
collection provides some time to 
reexamine the survey’s design and 
consider changes to enhance the relevance 
and quality of the SASS data collection 
effort. 

This paper focuses on one aspect of the 
survey’s design— survey depth— and 
addresses three major questions: 

(1) At what level should estimates be 
provided? Are state- and affiliation- 
level estimates useful enough to 
justify the large sample size they 
require? Are the oversampled 
groups of schools and teachers still 
important to single out? 



(2) From whom should data be 
collected? Should any new 
categories of respondents be added 
to the survey? 

(3) How much data can reasonably be 
collected from each respondent? 
What options are available to 
expand data collection without 
overburdening respondents? 

The paper starts by describing the context 
within which choices about survey depth 
must be made. It examines the purpose of 
SASS, identifies some important emerging 
policy issues and their implications for the 
survey, and describes the users and uses of 
the data. The paper then takes each set of 
questions in turn, discussing the issues 
they raise and making specific 
recommendations on how to address them. 



Context for Considering Survey 
Depth 

To make appropriate decisions about 
survey depth, we must first consider the 
purpose of the survey, who uses (or might 
use) it, and to what end. Although the 
future scope of SASS is currently being 
debated, some assumptions were necessary 
in order to make recommendations about 
survey depth. For the purposes of this 
paper, I have assumed that the general 
goals and structure of the survey will 
remain intact, but have also taken into 
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account emerging policy issues that might 
appropriately be monitored through SASS. 
Thus, although this paper focuses on 
survey depth, it unavoidably strays into the 
realm of survey scope. 

Purpose of the Schools and Staffing Survey 

SASS was designed to inform Congress, 
the Department of Education and other 
federal agencies, state education agencies, 
local districts, educational associations, 
and the larger education community on 
four major topics: (1) teacher supply and 
demand; (2) school conditions and 
programs; (3) the characteristics of the 
elementary and secondary teaching force; 
and (4) the characteristics of principals 
(National Center for Education Statistics 
[NCES], 1995b). In the mid-1980s, when 
the first SASS was designed, many 
analysts were predicting a serious 
impending teacher shortage, particularly in 
math and science. An important purpose 
of SASS, therefore, was to monitor the 
flow of teachers nationwide as they moved 
in and out of the teaching work force in 
ways that earlier elementary and secondary 
data collections were unable to do. 

After more than a decade of concentrated 
attention to school reform at the federal, 
state, and local levels, the four general 
topics mentioned above remain as relevant 
as ever. The focus of some of the 
questions related to them has changed, 
however. For example, when the 
predicted teacher shortages did not 
materialize, researchers and policymakers 
shifted the emphasis of their work from 
the absolute number of teachers to more 
complex issues such as the supply of 
teachers in specific fields, the distribution 
of qualified teachers across various types 



of schools and subject areas using various 
definitions of “qualified,” the match 
between qualifications and assignments as 
indicators of shortages in various fields, 
difficulties filling new positions, the 
adequacy of the supply of minority 
teachers, and the relationship of school and 
teacher characteristics to attrition. 

Although teacher shortages may not exist 
in the aggregate now, monitoring supply 
and demand remains a worthy goal 
because shortages could still become a 
problem in the future. In 1993-94, about 
one-quarter of all newly hired teachers 
were not teaching the year before (Henke, 
Choy, & Geis, 1996). Thus, the size of 
the pool of potential teachers is uncertain 
and may be influenced by many factors 
such as teacher salaries, the attractiveness 
of alternative careers, and the general state 
of the economy, to name a few. 

Like supply and demand, school conditions 
and programs and the characteristics of the 
school work force still command attention, 
although the focus of some of the 
questions has changed in these areas as 
well. SASS has been updated to reflect 
this. Compared with previous 
administrations of the survey, the 1993-94 
SASS has more questions on teacher 
education and certification, professional 
development, the influence of various 
participants in school decision making, and 
the variety of programs and services 
schools offer. 

In summary, SASS has been well 
structured to capture information on 
enduring issues and has been modified 
appropriately to reflect changing policy 
concerns through the early 1990s. Now, 
however, after more than ten years of 
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reform initiatives, a hard look at the 
information being gathered is necessary; it 
is important to monitor the implementation 
and impact of these initiatives, and to try 
to anticipate what people will want to 
know about schools and staffing five years 
from now when the data from the next 
SASS administration are released. No one 
would argue that an ongoing national 
survey monitoring conditions in the 
schools should be redesigned to follow 
every educational fad. Nevertheless, 
policies and priorities do change over 
time, and it is important for SASS to 
remain relevant and timely. 

Emerging Issues 

As we head toward the 21st century, some 
of the major issues that education 
researchers and policymakers are focusing 
on include the following: 

• What students are taught 
(curriculum frameworks, course 
content, graduation requirements); 

• How students are taught 
(instructional practices, use of 
technology); 

• How their progress is evaluated and 
what are the results (tests, 
portfolios); 

• Who is teaching, how they prepare 
for this responsibility, and what 
knowledge and skills they bring to 
bear (sources of teachers, minority 
teachers, content of teacher 
education, professional 
development); 



• How students are prepared for 
school (preschool education, early 
childhood experiences), and how 
they are assisted in making the 
transition from high school to 
further education or work (school- 
to-work transition programs, tech- 
prep programs); 

• How schools are organized and 
managed, including who makes 
what kinds of decisions (site-based 
management, decision-making 
processes, choice, vouchers, 
magnet schools, charter schools, 
networks). 

Implications for SASS 

There is much we want to learn about 
these reforms. However, it is important to 
pause and think about what SASS can and 
cannot do well given its overall size and 
design. 

SASS should include only items that are 
readily measured and that have a readily 
understandable definition. At the recent 
NCES-sponsored conference on possible 
future directions for NCES, Jennings and 
Stark (1995, pp. 4-5) note this as a barrier 
to collecting information on school reform. 
They suggest establishing an advisory 
committee made up of individuals with 
widely varying views on education reform 
so that they can help determine what to 
study and develop common definitions. 
However, even if common definitions 
could be agreed upon through such a 
process, we certainly cannot assume that 
these definitions will mean the same thing 
to all the individuals answering questions 
about them. This is especially true for 
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reforms that are new and not yet clearly 
defined. 

For example, the 1993-94 SASS asked 
schools if they offered a “tech-prep” 
program. Tech prep was defined as 
“vocational-technical instruction in the last 
two years of high school designed to 
prepare students for two years of 
vocational instruction at the postsecondary 
level.” With this definition, almost any 
school with a vocational program could 
respond affirmatively to this question, and, 
in fact, 56 percent of public schools with 
12th grades reported that they had such a 
program (Henke et al., 1996). However, 
“tech prep” has now come to mean 
something much more specific, with the 
critical defining feature being some type of 
formal articulation between secondary and 
postsecondary programs. Consequently, 
not much has been learned from this 
question. This example also illustrates the 
dangers of trying to measure something 
too soon, before there is a common 
understanding of what it is. If a reworded 
tech-prep question is included in the next 
SASS, the percentage of schools with tech- 
prep programs will probably appear to 
decrease, which would not be an accurate 
depiction of what is actually going on. 
NCES must guard against adding questions 
on “hot topics” that do not yet have 
commonly understood or easily 
communicated definitions. 

SASS can only measure things that are 
widespread. National surveys are not 
useful for providing information on 
reforms that involve relatively small 
numbers of schools or teachers, as new 
structures and practices often do. For 
example, some schools are supporting each 
other in the reform process by establishing 



networks of schools with common goals, 
such as the Coalition for Essential Schools. 
In a nationally representative sample, 
however, you will not pick up enough 
schools to say anything useful about 
schools that belong to this network or to 
compare them with schools that belong to 
other networks. For now, if you want to 
study these networks, the best approach 
would be to start with the list of 
participating schools. Similarly, one day 
we might want to track the movement of 
teachers certified by the National Board 
for Professional Teaching Standards 
(Mandel, 1996, p. 3-30). At present, 
however, there are fewer than 1 ,000 such 
teachers. You might not sample any of 
them in a national survey, and certainly 
not enough to make any comparisons to 
other teachers. 

Of the different types of reforms and 
changes occurring in schools and the 
teaching profession now, some are more 
suitable for including in SASS than others. 
It is relatively easy to ask who, what, or 
how many. Much harder to answer, 
however, are “how” questions. Thus, it is 
fairly easy to ask about professional 
development activities (how many 
workshops of a certain duration a teacher 
attended, for example, and in what subject 
areas) or teacher education (measured in 
terms of courses, majors, minors, or 
degrees). It is also relatively easy to 
measure how many students participated in 
certain programs (like tech prep), or how 
many plan to apply to college. It is also 
feasible to ask about the existence of 
certain policies that have readily 
understandable meanings (for example, 
choice programs or vouchers). 
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Much harder to measure are organizational 
management and issues such as how 
decisions get made. For these, national 
data collections such as SASS are less 
useful. In 1993-94, for example, teachers 
and principals did not agree on how much 
influence teachers have on certain types of 
decisions. Although this is worth 
knowing, it does not give us a clear 
picture of what was going on in schools. 
Because organizational processes are so 
complex, I therefore disagree with Baker 
(1996, p. 4) that SASS would be very 
useful for studying these processes, and 
believe that smaller studies using 
interviews would be more appropriate. A 
further complication is that the teachers in 
SASS are not representative of the teachers 
in their school. In fact, their perceptions 
about how their school works could differ 
quite markedly from those of their 
colleagues. Samples for schools are too 
small, for example, to permit HLM 
analyses despite the hierarchical structure 
of the surveys. 

Implications for Survey Depth 

1 . NCES should take advantage of the 
fact that SASS, because of its linked-survey 
design, is uniquely positioned to monitor the 
extent to which various types of reforms are 
present in schools and in the classroom. 

SASS is the best vehicle NCES has for 
following how school reforms have been 
implemented and for determining whether 
or not state-initiated reforms have filtered 
down to the district, school, and 
classroom. States can develop wonderful 
curriculum frameworks, for example, but 
if teachers do not actually use them, we 
cannot expect to see their effects on 
student learning. SASS can ask teachers 
questions about instructional practices, and 



we can use that information to examine 
implementation patterns by school and 
district characteristics. 

2 . More information is needed to 
describe what goes on at the classroom level. 
In recent years, there has been increasing 
recognition that the quality of resources 
has to be measured at the classroom level. 
Teacher quality and opportunities to learn, 
not just district or school spending, which 
have traditionally been used as indicators 
of quality, are key to educational success, 
and therefore their distribution across 
schools and states is of major national 
interest. The implication for SASS is that 
more information about what is going on 
in the classroom is needed. 

Of the current SASS participants, teachers 
are the best source of information on their 
qualifications and on what is going on in 
the classroom. They can be asked, for 
example, what professional development 
activities they have participated in, if they 
are using a state curriculum framework, 
what assessment practices they are using, 
and how technology is used in the 
classroom. Self-report data, even with 
such straightforward items such as degrees 
earned and dates, pose problems (Chaney, 
1994, pp. 8-12) but at the national level is 
the best we can do. On a smaller scale, 
self-report data can be combined with 
other types of data collection such as video 
or case studies. 

Data Users and Uses 

SASS has been targeted to a wide 
audience, including policymakers at all 
levels, researchers, and the general public, 
and some of their data needs differ. 
Researchers, while interested in national 
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data, typically do not need the large 
sample sizes required to support state-level 
estimates. Education administrators, on 
the other hand, want to know about 
variation across states so that they can 
make decisions about funding, reform 
initiatives, and staffing (NCES, 1994, p. 
1). Researchers are typically trying to 
establish links between educational 
structures and practices and student 
outcomes, while administrators and 
policymakers are more frequently 
interested in monitoring the 
implementation of proven or desired 
policies and practices. 

From a survey depth perspective, the 
important question is whether the state- 
and association-level data are used widely 
enough to warrant continuing to produce 
estimates at this level. It is difficult to 
know the extent to which state data are 
used. NCES has published some state-by- 
state SASS data in the Statistical Profiles 
and in SASS by State, produced jointly 
with the Council of Chief State School 
Officers and Horizon Research, as well as 
in the Digest of Education Statistics and 
The Condition of Education, but it is 
reasonably safe to say that these 
publications do not provide everything a 
state might need. 

For state-level data to be really useful, 
states need to have the capability to 
analyze SASS data themselves so they can 
tailor comparisons to their specific needs. 
The widespread availability of personal 
computers and appropriate software and 
NCES’s practice of providing large data 
sets on CD-ROM have made this 
increasingly feasible. Moreover, a number 
of states have obtained the data themselves 
or requested analyses from the National 



Data Resources Center (NDRC). The data 
would be even more accessible if NCES 
were to create a Data Analysis System 
(DAS) for SASS, such as that developed 
by Dennis Carroll for some of the other 
NCES databases. The DASs have allowed 
users with very limited computing capacity 
and technical knowledge to create their 
own cross-tabulations. 

The important point here is that rapid 
advances in technology are opening 
exciting possibilities for innovations in 
data collection, management, and 
distribution, and we are moving toward a 
closer connection between data producers 
and data users. 1 For example, Statistics 
Canada recently held a symposium entitled 
From Data to Information that explored 
topics such as the role of the customer in 
“co-producing information,” the 
integration of data collection and analysis, 
advances in analytic techniques of 
customers, and the privacy and 
confidentiality issues associated with new 
customer/supplier partnerships (Scheuren, 
1996). These trends suggest more, rather 
than less, demand for and use of state and 
other small area data in the future and a 
need for NCES to pay close attention to 
how SASS data are being or might be 
used. In the past, users of NCES data 
have been fairly well known to NCES 
staff, but as use becomes more dispersed, 
NCES may have to actively seek out the 
users to find out who is using what data 
and for what purpose. 

This discussion of policy issues and data 
users provides the context within which to 
proceed with the discussion of survey 
depth. In the rest of this paper, I return to 
the questions posed at the beginning: (1) 

At what level should estimates be 
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provided? (2) From whom should data be 
collected? and (3) How much data can we 
collect from each respondent? In each 
case, the issues are discussed first, 
followed by specific recommendations. 

At What Level Should Estimates 
Be Provided? 

Issues 

The 1993-94 SASS sample design 
provides estimates at both the national and 
state levels for public schools and 
teachers, and at the national and affiliation 
levels for private schools and teachers. In 
addition, the sample design provides 
state/elementary and state/secondary 
estimates; estimates of public and private 
libraries and librarians at the national 
level; and national estimates of public and 
private school students by grade level and 
urbanicity. To improve the accuracy of 
estimates for certain sampling units that 
were relatively small in number but 
important for policy reasons, SASS 
oversampled schools in which more than 
19.5 percent of the students were 
American Indian/ Alaskan Native, included 
all Bureau of Indian Affairs (BIA) 

Schools, and oversampled bilingual/ESL, 
Asian/Pacific Islander, and American 
Indian/ Alaskan Native teachers. 2 

The level at which estimates are provided 
has implications for cost, response rates, 
and the precision of estimates. In the next 
section, the implications of each of these is 
discussed in turn. 

Cost. Since the decision to provide state- 
and affiliation-level estimates affects 
sample size, it has a direct impact on the 



cost of administering the survey. To 
provide estimates at this level, the 
1993-94 SASS included approximately 
13,000 schools and 68,000 teachers. 

NCES could substantially reduce the 
sample size and therefore the cost of the 
survey if SASS provided only national 
estimates. Moreover, the reduction in 
sample size could free up resources to 
enhance the survey in other ways, such as 
covering more types of school staff, 
conducting the survey more often, or 
increasing the amount of information 
collected in each survey. The important 
question, then, is: Are state- and 
affiliation-level data useful, or should the 
sample size be reduced and the resources 
reallocated? Are there better ways to 
spend the resources allocated to SASS than 
having such a large sample? 

Response rates. Survey depth may also 
affect response rates, although the precise 
impact, particularly of marginal changes, 
is not easy to predict or measure. State- 
and affiliation-level estimates might 
contribute to reduced response rates 
because the larger the sample, the greater 
the probability that a given respondent will 
be surveyed more than once over time. 
Since respondents may view being asked to 
participate repeatedly as a significant 
burden they may be more likely to decline 
to participate a second or third time. 

Providing state- and affiliation-level 
estimates, requiring a larger sample, could 
jeopardize district response rates. In the 
1991 LEA pretest, for example, response 
rates were significantly lower for overlap 
compared with nonoverlap LEAs (84 
percent compared with 95 percent), 
suggesting a reluctance to participate 
repeatedly (NCES & U.S. Bureau of the 
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Census, in press, p. 39). This problem 
could worsen over time as the amount of 
overlap increases with repeated SASS 
administrations, although the shift to a 
five-year as opposed to a three-year 
interval between surveys will undoubtedly 
mitigate the problem. Reducing the 
number of questions asked of districts (by 
dropping them or shifting them to the 
school questionnaire) might also improve 
response rates for overlap schools. 

Repeated participation does not seem to 
have had a negative effect on the school 
response rate. In the 1993-94 SASS 
administration, the response rate for 
overlap and nonoverlap schools was about 
the same among public schools (92 
percent). Among private schools, overlap 
schools actually had a higher response rate 
than nonoverlap schools — 83 percent 
compared with 88 percent (NCES & U.S. 
Bureau of the Census, in press, pp. 

39-40). 

Although there is no evidence to prove 
this, it seems reasonable to hypothesize 
that providing affiliation- level estimates 
would affect the private school response 
rate positively rather than negatively. 
Private schools are not a “system” and 
typically have less interest than public 
schools in making intrasector comparisons. 
The affiliation-level estimates allow private 
schools to define a more relevant 
comparison group than all private schools. 
For example, Montessori schools can 
compare themselves with other Montessori 
schools, and Catholic schools with other 
Catholic schools or with other religious 
schools. Without the affiliation-level 
estimates, private school associations 
might not be willing to endorse the survey. 
The response rate for private schools is 



already lower than for public schools— 83 
percent compared with 92 percent in 
1993-94 (NCES & U.S. Bureau of the 
Census, in press, p. 4). Without 
association endorsements, the response rate 
might be even lower. 

Precision of estimates . Finally, survey 
depth has implications for the precision of 
estimates. The optimum sample design for 
national estimates is different from that for 
state-by-state comparisons. For the best 
state comparison, schools and teachers 
should be sampled in proportion to their 
numbers in each stratum for national 
estimates, but equally among states. Since 
both types of comparisons have been 
important, the sample has compromised on 
these goals (NCES, 1991, pp. 9-10). The 
precision of the national estimates could be 
improved by abandoning state-level 
estimates. 

The oversampling of certain types of 
teachers does not have cost implications 
because the oversampling does not change 
the total sample size. Larger samples of 
bilingual/ESL, Asian/Pacific Islander, and 
American Indian/Alaskan Native teachers 
are obtained by sorting the teacher lists so 
that adequate numbers of these types of 
teachers can be selected. However, 
oversampling does improve the precision 
of the estimates for these groups. 

Recommendations 

State- and affiliation-level estimates. I 
believe that a strong case can be made for 
continuing to support state- and affiliation- 
level estimates given the purpose of SASS, 
the nature of emerging policy issues, and 
the context in which school reform 
initiatives are taking place. First, we need 



state-by-state data to describe accurately 
the education enterprise in the United 
States. Education is primarily a state and 
local responsibility, and the current trend 
is away from federal involvement and 
uniformity at the national level, not toward 
it. Moreover, many important aspects of 
schooling are typically controlled or 
heavily influenced by state policy, such as 
programs offered, curriculum, graduation 
requirements, the number of days per year 
and hours per day that schools must be in 
session, pupil/teacher ratios, expenditures 
per student, teacher certification 
requirements, and salary schedules, to 
name a few. Key features of the education 
system cannot be captured without state- 
level data. 

Second, the major current school reform 
efforts are being implemented at a 
subnational level, even those initiatives 
originating at the federal level. The 
federal School-to-Work Opportunities Act, 
for example, has provided the states with 
grants and given them much freedom to 
design their own programs. In fact, there 
is much more talk in Washington about 
block grants than about new federal 
programs. The reality is that most of the 
major reform efforts are being initiated at 
the state level. In the area of curriculum 
standards, for instance, states have moved 
with varying strategies and at different 
speeds, with differences across states as 
noteworthy as the similarities (Cohen, 

1995, pp. 11-12). To monitor reform, a 
state perspective is needed. 3 

Third, state-level data provide 
policymakers with information useful for 
making state policy, such as scheduled 
salaries. Although states can generate 
some of this information themselves 



through their own databases, SASS allows 
comparisons with other states, which is 
often hard to do when data do not come 
from a common source. 

Fourth, providing state-level data may 
build support for NCES data collections 
with federal legislators because their 
constituents will be happy. In a similar 
vein, providing affiliation-level data will 
make the private school associations 
happy, thus encouraging them to endorse 
the survey, which, as suggested above, is 
likely to improve the private school and 
teacher response rates. 

Finally, it would be a shame to eliminate 
state-level estimates just when the 
technologies being developed for data 
collection, management, and dissemination 
are leading toward increased usefulness of 
nationally collected data at subnational 
levels. 

The major argument against continuing to 
support state- and affiliation-level estimates 
is cost. Cost savings could be a strong 
argument if there were better ways to use 
the funds. The major pressure right now 
seems to be how to get more information 
from teachers. Eliminating state- and 
affiliation-level estimates would not do 
much to accomplish this goal because the 
chief difficulty is the response burden on 
individual teachers. 

Oversampled groups. Continuing to 
oversample schools with large American 
Indian/ Alaskan Native student populations 
and to oversample bilingual/ESL, 
Asian/Pacific Islander, and American 
Indian/ Alaskan Native teachers also seems 
appropriate. The issues that prompted the 
oversampling in the first place have not 
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disappeared. The only question would be 
whether other groups have greater priority, 
and this does not seem to be the case. 

The American Indian/ Alaskan Native 
student population is relatively small 
(about 1 percent of the total student 
population) and therefore would not be 
well enough represented in a national 
survey of schools and staffing to permit 
reliable generalizations about their 
characteristics. Yet there are policy 
concerns about the condition of education 
for these students as well as some 
systematic differences in the characteristics 
of these schools and staff compared with 
other schools and staff, and these bear 
monitoring (NCES, 1995a, p. 1). 

In the United States as a whole, 14 percent 
of all children aged 5 to 17 spoke a 
language other than English at home in 
1990, and 5 percent had difficulty 
speaking English. The percentage of 
school-age children with difficulty 
speaking English increased by 27 percent 
between 1980 and 1990 (Smith et al., 

1994, p. 130). Thus, it will continue to 
be important to monitor the adequacy of 
the supply of teachers to help these 
students. 

Because Asian/Pacific Islanders and 
American Indian/ Alaskan Native teachers 
made up such a small percentage of the 
total teacher population (1.1 percent and 
0.7 percent, respectively, in 1993-94) 
(Henke et al., 1996), it would be difficult 
to study them without oversampling them. 
Obtaining accurate data on minority 
teachers is important because the adequacy 
of the supply of minority teachers and 
their distribution among different types of 
schools remain crucial issues. 



From Whom Should Data Be 
Collected? 

Issues 

One of the major strengths of SASS is the 
integrated survey design that links schools, 
teachers, districts (for public schools) and, 
as of 1993-94, libraries, librarians, and 
students. Questionnaires are completed by 
districts, schools, principals, teachers, and 
librarians. Two issues are important to 
consider here. One is whether this is the 
correct structure. Are we getting all the 
information we want, or should we include 
other staff such as vice principals, 
department heads, or counselors? 

The other issue is whether we are asking 
the right questions of the right people. 
There are two reasons why we might want 
to change. One is to improve accuracy. 
For example, until the 1993-94 
administration of SASS, information on 
public school teacher benefits was 
collected from the district, which was 
asked to identify from a list of benefits 
those that were offered to teachers. 
Beginning in 1993-94, questions on 
benefits were shifted to teachers, who 
were asked what benefits they received, 
which was much more useful because it 
provided more accurate information and 
allowed direct comparisons with benefits 
received by principals. The second reason 
we might want to change the source of 
information is to reduce the response 
burden imposed on particular types of 
respondents. For example, if some 
questions currently on the district 
questionnaire could be shifted to the school 
questionnaire, the response burden for the 
districts could be reduced. 
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Recommendations 

Since schools employ a wide variety of 
types of staff, collecting information from 
principals and teachers necessarily 
provides only a partial picture of school 
conditions. Other staff who could provide 
information on school conditions include 
vice principals, department heads, 
counselors, and school superintendents. 
However, the major problem with 
including these types of staff is that their 
roles vary widely from school to school, 
giving them quite different perspectives. 
School superintendents would be a useful 
source of information on both state and 
district policies, but probably no better 
than whoever fills out the district 
questionnaire. It might be interesting to 
learn about superintendents’ backgrounds, 
education, and career paths, but this would 
not be a high priority given the ambitious 
goals already outlined for SASS. 

If extending the survey depth to collect 
more information about what goes on in 
the classroom is considered desirable, the 
question then arises as to who should 
provide the information, teachers or 
students. The current Teacher Followup 
Survey (TFS) survey contains a 
considerable amount of information on 
teaching methods collected from teachers. 
However, judgments on the usefulness of 
this information will have to wait until the 
data on classroom practices are evaluated. 
Self-reported data on this topic may prove 
valuable, but could prove to have 
significant limitations. This remains to be 
seen. Since the issue of whether to 
include students in SASS is addressed in 
another paper, it is not addressed here 
(Kaufman, 1996). 



Over the three administrations of SASS, 
NCES has switched the sources for some 
data (such as teachers’ benefits) to 
improve quality. Overall, it appears that 
the right questions are now being asked of 
the right individuals to maximize accuracy. 
However, since the district questionnaire 
takes the longest to complete, it is worth 
considering whether the burden on the 
districts could be reduced by shifting some 
questions to the schools. Some district 
policies would be known at the school 
level, such as the number of days in the 
school year or whether or not the district 
had a choice program (although schools 
would not know how many students 
participated). Collecting such data from 
schools could lead to the problem of 
reconciling data from multiple schools in 
the district. Also, it could irritate the 
school personnel filling in the 
questionnaire, who might wonder why the 
questions were being asked of them rather 
than the district. Finally, some 
information on district policies would only 
be available reliably from the districts, 
such as salary schedules, incentive pay 
policies, hiring criteria, and policies on 
retraining. On balance, there does not 
seem to be much prospect for shifting 
much of the response burden away from 
the districts. The most burdensome 
questions are those that require looking up 
numbers, such as student enrollments, and 
the district is the only source of this 
information. 

As a final point, it is worth keeping in 
mind that the appropriate level to collect 
data may change in the future with 
advances in data collection technology. 

For example, information on enrollments 
now collected from the schools may be 
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more easily obtainable from a district or 
even a state database. 



How Much Data Can We Collect? 

Issues 

Once a survey has been developed, the 
marginal cost of fielding an additional 
question is relatively low. It is very 
tempting to keep adding questions, but the 
response burden cannot be ignored. 
Estimated times for completing the surveys 
(printed on the questionnaire) were as 
follows: LEAs, two hours; public schools, 
one hour; private schools, two hours; and 
principals, 30 minutes. The time teachers 
took to complete the survey was not 
estimated, but the average time reported 
by public school teachers in 1993-94 was 
36 minutes. The overall burden on the 
principal may be considerably greater than 
30 minutes depending on how much of the 
school questionnaire the principal has to 
complete personally. In a small school 
with no other administrators, the burden 
for responding to the school questionnaire 
and developing the teacher list probably 
falls on the principal. 

A twist on this issue is that the types of 
questions may be more relevant than the 
length of time it takes to respond. 
Respondents may not object to completing 
a long questionnaire in which the questions 
are easy to answer, but may balk at a short 
questionnaire that requires them to look up 
numbers or that includes questions that are 
puzzling or difficult to answer. Thus, 
NCES should pay close attention to the 
types of questions asked as well as how 
long they take to answer. 



To date, overall response rates have been 
good, suggesting that the burden is not yet 
excessive. 4 However, the response rate to 
the initial mail survey (as opposed to the 
telephone follow-up) was considerably 
lower, suggesting that there may already 
be too many questions for some 
respondents. Although the response rate 
has been increased by telephone follow-up, 
reinterviews have shown that the responses 
obtained through the mail survey were 
more reliable (Bushery, Royce, & 
Kasprzyk, 1994, p. 7). 

Recommendations 

The major increase in response burden for 
new data collection to address emerging 
issues will fall on the teacher. For 
example, including all the information 
asked on the TFS in the regular SASS 
teacher questionnaire would greatly 
increase the response burden for teachers. 

A variety of options exist to address this, 
including the following: 

1. Assume this is not a problem, and 
add to the questionnaire. One could point 
to the high response rates attained in 
previous administrations of SASS and 
argue that more questions could be added 
without reducing the response rate. This 
is an empirical question, at least for now, 
and could be answered with a field test. 

2. Try to maintain the amount of time 
that it takes to complete the questionnaire 
while increasing the number of questions. 
Changes in technology may help make this 
possible. At some point in the not-too- 
distant future, most teachers will have 
access to and be able to use computers. 
Thus, it may be not only feasible but also 
very efficient to provide the questionnaires 
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to teachers in electronic form, at least as 
an option. A teacher might be able to 
move considerably more quickly and easily 
through the questionnaire if he or she did 
not have to pay attention to skip patterns. 
Electronic data collection from teachers 
and schools is already being investigated 
(Kasprzyk, 1994, p. 3). This could reduce 
the cost of data collection per teacher 
(making larger samples feasible) and 
improve accuracy (by eliminating skip 
pattern errors, for example). To what 
extent it would reduce response time 
enough to permit asking a lot more 
questions is largely unknown, but bears 
investigating. 

Also, as indicated above, it is not only the 
number of questions but also their 
difficulty that affects response rate. If the 
questions are interesting to teachers and 
easy to answer, they may not take much 
longer. 

3. Eliminate some of the current 

questions. It is very difficult to identify 
items that could be dropped from the 
teacher questionnaire, but a couple of 
suggestions are made here. Among the 
least useful seems to be the detail on what 
teachers were doing before they started 
teaching at that school (Questions 6-11). 
When they first taught and how many 
years of total teaching experience they had 
could be sufficient. It may also be 
unnecessary to include the detail on the 
number of courses ever taken in various 
fields because this is a lot of work to 
figure out and may not be very meaningful 
or accurate except for recent graduates. 
Moreover, this may be the type of 
question that causes teachers to give up. 
Since they could answer a lot of questions 
in the time it takes to dig up all that 



information, the questions on majors and 
minors may be enough. 

4. Do not ask all teachers all questions. 
This would require increasing the total 
sample size to continue state- and 
affiliation-level estimates for all data 
items. The division of questions would 
have to be done very carefully to ensure 
that you did not split data among 
subsamples that you want to examine 
together. 

5. Cycle some questions. For example, 
on one survey you might ask about 
instructional practices and on the next 
perceptions of school problems or decision 
making. This means that some questions 
would only be asked every ten years. In 
reality this might not be a serious loss. 
Comparisons of data from 1987, 1991, and 
1994 show considerable stability in many 
areas. However, policymakers and the 
general public increasingly expect up-to- 
the-minute data (i.e., ten-year old data 
have little credibility). 

6. Use the TFS to collect data for which 
national estimates are sufficient, to field test 
new data, or both. In 1994-95, the TFS is 
being used to question teachers extensively 
about their instructional practices. We do 
not know yet what the quality of these data 
will be, but if it seems worthwhile to 
collect similar information again, the TFS 
may be an appropriate permanent home 
simply because there may be too many 
questions to add to the regular teacher 
survey. The TFS is a good testing ground 
for new subject areas because the sample 

is small yet nationally representative. 

7. Use the SASS sample as a framework 
for more limited studies. Metcalf (1995), 
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for example, has explored the feasibility of 
incorporating experimental designs into 
NCES data collection methodologies. If 
everything we are interested in cannot be 
addressed in a national data collection, it 
might still be possible to use SASS as a 
sampling framework for additional, more 
specialized studies. 

More information is needed to know which 
of these options (or combinations of 
options) are most promising. In 
particular, we need to know how 
successful the TFS data collection on 
instructional practices turns out to be as 
well as the scope of the survey. 

Conclusion 

My major points can be summarized 
briefly as follows: 

(1) SASS has been an extremely 
valuable survey, and its current 
structure of an integrated set of 
linked surveys is useful and 
appropriate to meet the purposes of 
the survey and to provide 
information on emerging policy 
issues. 

(2) SASS is the best mechanism NCES 
has for monitoring the 
implementation and diffusion of 
many current school reform 
initiatives. 



(3) Given the current focus on 
measuring quality at the classroom 
level, SASS needs to reach more 
extensively into the classroom. 

This means obtaining detailed 
information on teacher training and 
professional development and the 
knowledge and skills teachers bring 
to bear (a direction in which the 
1993-94 SASS has already moved), 
and on instructional practices 
(beginning to be addressed through 
the TFS). 

(4) State- and affiliation-level estimates 
should be continued, as should the 
current oversampling of schools 
with large American Indian/ Alaskan 
Native student populations and 
oversampling of bilingual/ESL, 
Asian/Pacific Islander, and 
American Indian/Alaskan teachers. 

(5) No additional types of respondents 
should be added to SASS. 

(6) Because of the interest in focusing 
on the classroom, the increased 
response burden will fall mainly on 
teachers. Simply adding the TFS 
questions to the next full teacher 
survey may not be realistic. A 
variety of options exist to address 
this problem, but the results of the 
TFS survey and decisions on 
survey scope are needed before 
recommendations can be made. 
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1. Two papers presented at the conference on Future NCES Data Collection: Some Possible Directions 
address this issue. See Scheuren (1995) and Ligon (1995). 

2. This is somewhat of an oversimplification of the sample design. For more detail, see NCES and U.S. 
Department of the Census (in press). 

3. Jennings and Stark (1995) recommend that NCES consider studying education reform on a state-by-state 
basis and issue annual reports on state activities. 

4. In the public sector the weighted response rates in 1993-94 were 93.9 percent for districts, 96.6 for 
principals, 92.3 percent for schools, and 88.2 percent for teachers. In the private sector they were 87.6 
percent for principals, 83.2 percent for schools, and 80.2 percent for teachers (NCES and U.S. Bureau of 
the Census, in press, p. 4). 
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REFLECTIONS ON THE PAPERS PREPARED FOR THE SCHOOLS 
AND STAFFING SURVEY SEMINAR SERIES 

John Howard Burkett, Wilmington, NC 



Education : n. That which discloses to the wise and disguises from the foolish their lack of 
understanding. 

Ambrose Bierce, The Devil 's Dictionary 



Preface 

The following Reflections are offered as a 
layman’s contribution to the ongoing 
discussion and planning of a "new" 

Schools and Staffing Survey (SASS) by the 
National Center for Education Statistics 
(NCES) beginning in 1998-99. They are 
based on a review of commissioned papers 
from the perspective of an (allegedly) 
educated citizen, familiar to a degree with 
the work of SASS and NCES but 
decidedly not a technical expert in 
statistics or survey research. It is hoped 
that these Reflections will stimulate 
discussion on the future content and design 
of SASS. 

In what follows some specific criticism of 
points made in the papers will be ventured. 
In some (if not most) cases, these 
criticisms may be misplaced, betraying the 
author’s nonexpert standing. If such there 
be, these mistakes will indicate where a 
"layman" has trouble with understanding 
just what the experts are trying to say. 

But the ultimate "test" of the work of a 
public statistical agency is its ability to 
communicate clearly to the public 
information relevant to the issues of 
concern, in this case how the nation’s 
education is faring. Thus, these criticisms 



are presented "baldly" as signs of the need 
for greater clarity. 

The overall aim, however, is not to argue 
specific points but to try to explore a 
"global" perspective on what NCES might 
do with SASS in light of the extensive 
technical work done on SASS in the past 
and in these commissioned papers. In this 
aspect, the author will probe some 
assumptions and proposals presented in the 
papers. Such probing is not intended to be 
definitive but "framing," in the sense of 
identifying major points of agreement and 
disagreement, and of suggesting lines of 
further inquiry and possible resolution of 
contentious issues. Above all, the 
Reflections are a starting point, neither 
conclusions nor definitive 
recommendations . 

Finally, the author apologizes if any of the 
writers of the commissioned papers 
believes his or her work has been slighted 
or maligned in this Reflections. The 
response is simply that the task here is 
general. It should be clear that a great 
deal has been learned from these papers 
and that such value as these reflections 
may have is due in large part to that 
learning (or misunderstanding). 
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SASS and the Public 

As is well known, and universally 
acknowledged by the authors of the 
commissioned papers, the data needs in 
education have expanded almost 
exponentially in the last decade and a half. 
While there has been a recognition of the 
need for statistics (literally "state 
information") since the founding of the 
nation, this need has been largely confined 
to small groups of people. And, as the 
early resistance to a census (and the latest 
complaints about it) indicates, the public 
generally has shown less enthusiasm for 
statistical enterprises. In spite of 
Americans’ love of information (witness 
the proliferation throughout our history of 
atlases, encyclopedia, etc.) development of 
thorough statistical surveys has always 
proved a difficult sale. 

There has been, I believe, a sea-change in 
attitudes toward the importance of statistics 
in education. Late this winter, the New 
Hanover County Schools in North Carolina 
issued its first "Report Card"— a 14-page 
newspaper section in the Wilmington 
Morning Star prepared by the school 
district, giving detailed information about 
the school district and about each school 
within the district. Comparisons were 
made of each school to the district 
averages for each type of school 
(elementary, middle, and high) as well as 
comparisons of the district to both state 
and national data in some areas. 

Though obviously limited in many ways, 
the "Report Card" was extensive and, most 
importantly, has struck a responsive chord 
in the community. I was particularly 
taken by a letter to the editor, praising the 
paper for publishing the publication and 



noting "I have always wanted to know how 
my children’s school was doing and now I 
am beginning to find out." While the 
usual issues of school politics dominate the 
upcoming school board elections in New 
Hanover County, I have been struck by the 
fact that in public forums some citizens 
have asked questions stemming from the 
release of the "Report Card. " 

The near identification of improvement in 
education with enhanced economic well 
being (if not survival) is now 
commonplace. The entire "Ideas" section 
of the Sunday, April 21, 1996, 

Greensboro, NC News & Record was 
devoted to education, under the general 
heading "Will future ’State of the South’ 
rise? Our fate awaits school bells." While 
some of us may view such claims as far 
too simplistic, the plain fact is that in 
some fundamental way the claims have 
taken hold and the public is demanding 
that schools do a better job. And in all 
this, the value of good statistical 
information is being recognized. 

Herein, of course, lie grave difficulties. 

The public (including you and me) is 
fickle. Worse, in education, the most 
outstanding feature is the rise and fall of 
fads. We all want the solution, and, 
especially if it concerns our own children, 
"We want our Maypo and we want it 
now." Statisticians and serious social 
science researchers are painfully aware of 
the dangers of responding merely to the 
latest fad. Instant polling and immediate 
reporting of political trends clearly 
demonstrate how even well-designed 
instruments yield misleading information. 
And given the rapidity with which 
education fads change, we know that 
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designing surveys to provide that Maypo 
now is worse than useless. 

But the fact remains that there is a public 
need for timely and reliable and valid data 
on education. The early success of SASS 
in responding to a concern ("teacher 
shortages") demonstrates that a survey 
program of real quality can arise from an 
immediately pressing need and advance the 
public understanding of the issue. Such 
work can advance that understanding and 
open new avenues of inquiry. 

My first point, then, is that there is a 
pressing public need for more information 
on just how schools are faring. Though 
seemingly trite, this point has, I believe, 
an important implication for thinking about 
SASS: attention must be paid to what the 
public wants and needs to know. 

Well, of course, you may reply, but so 
what? Put as strongly as possible, in all 
our deliberations, the ultimate touchstone 
is whether the information will make sense 
to the public! Put another way, we need 
constantly to ask, "Would I, as a parent 
and citizen, gain information from SASS 
that will make a difference in how I 
approach my children’s education and that > 
of all the children in my community?" 

A noble thought, but how can we seriously 
address the public at large? Notice that I 
have insisted that we take making sense to 
the public as a touchstone for work in 
SASS. By that I do not mean that SASS 
should become the USA TODAY of the 
education world. For while public 
concerns are the basis for the work, it is to 
the public policymakers that the work must 
be primarily addressed. It is the 
policymaker, the politician (in the best 



sense of that term), who must respond 
directly to the public needs and desires. 
And it is thus to that audience we must 
turn in asking what SASS can be as an 
important part of the public debate about 
education. 

But, again, I want to insist that keeping in 
mind what the public wants and needs to 
know are central to the enterprise. That 
danger in focussing on policymakers is that 
we fail to remember that they are 
especially subject to the whims of public 
opinion and pressure groups. Without a 
touchstone, we risk merely pleasing the 
"masters" without addressing the public 
needs. 



Policymakers as Audience 

While there appears to be general 
agreement that policymakers are the 
primary audience for SASS, it is not clear 
just what that entails. Policymakers no 
less than the public at large primarily want 
to do something— now. As NCES’s 
experience with the National Educational 
Goals Panel shows (at least through 1994), 
policymakers are impatient with the 
limitations of statistical information— both 
in scope and timeliness. In many 
instances, NCES was able to persuade the 
Panel that these limitations be accepted. 
That experience (at least for me) 
demonstrates the value of carefully 
designed surveys and studies and the 
ability of policymakers to grasp that value. 

What is needed, I suggest, is the kind of 
approach set forth in general terms in 
Professor Boe’s paper (pp. 185-212) for 
this seminar. In it, you will recall, he 
argues for three "guidelines" for selecting 
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aspects of the educational enterprise to 
include in SASS data collections. I 
paraphrase from Boe’s paper (p. 190): 

(1) Select topics that have been subject 
to major policy action in the recent 
past that focus on "fundamental 
aspects of schooling. " 

(2) Select topics that are subjects of 
recent analyses and debates but not 
of major policy action. 

(3) Select topics of major public 
concern. 

These guidelines and Boe’s discussion of 
them capture the core of what I think 
should be SASS’s approach. His argument 
is simple and straight-forward: one cannot 
do everything but one can take what are 
basic public concerns, as reflected in 
policy actions and debates, and in general 
public discussions. As Boe notes (p. 192), 
what needs to be focussed on are those 
aspects amenable to policy intervention. 1 

Boe also reminds us of the importance of 
distinguishing between monitoring what is 
happening in education and specifically 
advocating any particular policy action 
(pp. 187-189). While a well-rehearsed 
slogan of the work of NCES, this 
distinction is often lost on the audience and 
even occasionally within the statistical 
community. 2 Frustrating as it may be, the 
desire to satisfy specific policymakers’ 
concerns must be resisted. Professor Boe 
offers good counsel on this issue. 

The upshot here is that something like 
Boe’s classification of fundamental aspects 



of schooling must be explicitly and 
carefully set forth. I do not argue for or 
against the particulars he sets forth but for 
the general approach and especially for the 
need to keeping such a classification at the 
forefront of thinking about SASS. 



Which Policymakers? 

Boe argues that policymakers at all levels 
should be the primary audience. In many 
ways there is little to disagree with in his 
claim. However, as Choy notes, 
"Education is primarily a state and local 
responsibility, and the current trend is 
away from federal involvement and 
uniformity at the national level, not 
towards it" (p. 221). The situation is 
further complicated by the fact that, 
among the states, there is great variance in 
the level of state versus local control in 
education. 3 The fact remains, I believe, 
that it is primarily state and local 
policymakers that should be the focus. 

If this be conceded, the suggestion in 
Blank’s paper needs to be taken seriously, 
namely, that NCES should work closely 
with state education information personnel. 
He draws a useful parallel to the way in 
which the Common Core of Data was 
improved with the Cooperative Statistics 
program. In any event, it seems to me 
that the value of SASS will ultimately be 
realized only if it addresses education at 
the state and local level. The federal 
needs must be secondary except in so far 
as a federal role in providing some level of 
uniform national data so that states can 
make meaningful comparisons. 
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What about Educators? 

If SASS is truly successful in providing a 
rich picture of schools and staffing, surely 
this information would be of value to those 
who formulate and implement specific 
educational policies. Blank urges that data 
must be of more use than just for 
monitoring the enterprise (p. 166). His 
argument is mainly based on the practical 
considerations of retaining teacher/ 
administrator cooperation in data 
collection. If we want the best data we 
will need that cooperation, so his point is 
well taken. 

There is, however, a danger in 
concentrating on this point in the general 
design of SASS. That danger parallels the 
one mentioned earlier about following 
mere fads in the public concerns. While 
appreciating the practical argument of 
securing participation (and surely his 
suggestions about communications with 
participants are sound), I caution against 
taking this point as a central focus. 

Instead, we need to think about the 
usefulness of SASS for teachers and school 
administrators, but more in terms of what 
such a survey would provide for research 
studies that can be of use to teachers. 

(I do want to note that if teachers are seen 
as a more direct audience than I have 
argued, Stodolsky’s paper raises some 
important issues about what should be 
involved in such a survey. Of special 
import is her account of needing to know 
about what teachers really understand 
reform to be. Perhaps for policymakers 
including something about teachers’ 
understanding of reform initiatives would 
be of value. Note, too, her point about 
the diverse understanding of the terms in 



which reforms are discussed— a point that 
parallels the one Choy makes as noted 1 .) 

Researchers and SASS 

In the argument above, I have stressed the 
view that SASS is primarily for education 
policymakers, and particularly for state 
and local authorities in education. But, it 
should be noted, SASS (and indeed all of 
NCES’s work) has always been conceived 
as providing data for educational research 
as well as for policymaking. 

In some ways there is a false dichotomy 
here. Data useful for policy debate and 
formulation are not devoid of value for 
research purposes. And clearly, research 
can inform the means for collecting and 
analyzing data for policy uses. More 
importantly, I think it important to 
distinguish between designing a survey for 
informing policymaking and designing one 
for research purposes. Large scale 
surveys are not essential for research, 
though their existence provide a context 
for it. For example, as Stodolsky notes, 
large-scale surveys have been valuable but 
finally are no substitute for observational 
studies and other research activities in 
examining curriculum and instruction (p. 
5). On the other side, the importance of 
using research to design an effective 
survey is unmistakable. 

To consider ways in a large-scale survey 
that research can be conducted is of course 
sensible. But not at the cost of limiting 
survey data to the research hypotheses. 
Thus, while Stodolsky argues that to 
understand the new developments in 
teaching ("the constructivist approach") 
SASS should include elements to examine 



235 



240 



this phenomenon (or link with other 
studies), to do so would, I am concerned, 
muddle the role of SASS as a general 
monitor. Something like Boe’s distinction 
between policy monitoring and policy 
advocacy needs to be made here. 

Sometime we just must say no. 

In several of the papers mention was made 
about how limited has been the research 
work based just on SASS. I am not 
certain that this fact constitutes a major 
criticism. I acknowledge there are 
technical issues about how readily the data 
in SASS can be linked to other data, etc., 
but I do not see these as of primary 
concern in the sense that these needs 
should shape major directions of SASS. 4 
If the earlier discussion has any merit, the 
value of SASS for policy monitoring far 
outweighs any direct research limitations. 
And, as suggested, we may not have a real 
either/or here. 



The Big Issue 

The above discussion has left one major 
issue hanging and gives an air of 
"abstractness" to the discussion. While 
there are many aspects of schooling that 
are of interest to the public, the heart of 
the matter is whether students are learning 
what supposedly they are being taught. 
That is, student achievement, the outcome 
of prime importance in education, has not 
been discussed. 

To lay the cards on the table, though one 
can talk about the many outcomes that 
schooling may have, it is for the purpose 
of formal academic instruction that schools 
have been established. Some of us are 
prepared to argue that society has gone 



astray in creating too many nonacademic 
roles for schools to perform. Robert 
Spillane has often argued that if you could 
get all the other roles out, the school 
would do much better in their academic 
mission. In any event, any serious 
redesign of SASS must address ways to 
explore ways to relate the conditions in 
schools and staffing with academic 
achievement. 

What parents and public want to know is 
how well are the schools doing in 
academic achievement. And, as all our 
seminar writers seem to agree, a well- 
designed SASS can be an important 
component of the database to answer this 
question. Further, with such information 
about the conditions of schools and 
staffing, richly elaborated, our knowledge 
of "what works" in teaching and learning 
could be greatly enhanced. 

Kaufmann has explored ways that SASS 
could be shaped and/or related to NAEP, 
NELS, and other studies of student 
outcomes. Detailed discussion of his 
proposals is beyond my competence. 
McLaughlin’s Comments on Kaufmann 5 
deserve our attention, and I eagerly await 
that discussion. 

One issue raised by McLaughlin needs 
emphasis: the linkages that are being 
developed through NAEP and developing 
state assessment systems (McLaughlin, 6 
pp. 4-5). Consistent with the position 
Blank takes, McLaughlin’s proposal for 
linking state-produced school-level 
achievement data with SASS data on those 
schools is promising. Clearly, the states 
are now forging ahead with reforms and 
data collections including achievement 
data. SASS can offer information that 
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provide the states with contextual 
information that is genuinely useful to 
states. The key here, as McLaughlin 
stresses, is that the focus must be on 
providing "school dynamics data to states 
for use in research, by states [emphasis 
added] ..." (McLaughlin, 7 p. 5). 

A key technical question that must be 
settled is whether McLaughlin is right that 
aggregate and cross-sectional student data 
is sufficient for addressing important 
SASS-related questions concerning student 
achievement. If he is, the approach he 
proposes is worthy of pursuit. 

One issue to keep in mind, especially 
when addressing an educated citizen who 
wants to see schools improve: when most 
parents express concerns, they are 
fundamentally focussed on how his or her 
child or children are doing. Nothing in 
SASS can answer that concern directly, of 
course. But SASS, in the context of 
developing state assessment and reporting 
programs, can provide useful information. 
Forging links between SASS and state 
agencies, as both McLaughlin and Blank 
suggest, not only provides a practical 
reason for cooperation with data collection 
but makes SASS an integral part of a truly 
national data system for education. 

Some Passing Thoughts 

I do not want to slight papers not 
mentioned in the above discussion, but my 
purpose has been to focus on what I take 
to be the heart of the matter. Let me 
make a few comments about some of the 
other papers, noting some points of 
interest and/or concern. 



Fulton and Gilford each argue for 
some detailed considerations of 
distinct aspects of schooling- 
technology and professional 
development activities in school, 
respectively. Without discussing 
the details of their presentations, let 
me suggest that these topics 
represent just the sort of items 
about which we need the prior 
thought about how fundamental (in 
Boe’s sense) these activities are to 
an understanding of what is 
happening in schools. Is technology 
just a fad? More importantly, 
without linkage of SASS to student 
achievement data, information 
about technology is vacuous. 
(Without a doubt nearly everyone 
now looks to technology as a kind 
of deus ex machina in education, 
but there are a few of us Luddites 
left.) Professional development 
activities strike closer to home in 
my view, and Gilford’s suggestions 
may prove useful in certain aspects 
of the teacher survey. 

Ross’s paper demonstrates my 
concern about a too narrow focus 
on particular federal programs and 
reform that, as his own presentation 
makes plain, is understood in so 
many different ways. To be sure, 
there have been federal programs in 
support of magnet schools, but I do 
not see much value in any detailed 
survey of this phenomenon in 
SASS. One place that some of the 
concerns herein expressed might be 
explored is in the resource aspects 
of the survey, especially if some of 
what Chambers suggests prove of 
use. 
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Chambers’ paper reminds us that 
merely gathering fiscal data may 
offer us very little real information. 
Dollar allocations are in themselves 
sterile. His suggestion to tie those 
dollars to real activities is on the 
mark. 

Zheng argues for the continued use 
of questions about teacher and 
principal perceptions of school 
conditions. But Chambers reminds 
us that objective measures of 
events, activities, and behaviors are 
far better sources of information. 
After all, as Chambers notes, "If 
schools do not maintain reporting 
mechanism on certain student 
behaviors or activities, then one 
might question whether or not the 
activity is really a serious problem" 

(p. 182). 

Baker’s paper deserves much more 
attention than I have given it. I 
have not yet sorted out all the 
aspects of his claim a new view of 
organizations would provide. 
Intuitively, the idea of looking at 
organizational processes rather than 
structures seems right. But, with 
Choy, I worry that the complexity 
of these processes would render 
any general survey data so general 
as to be useless. Baker’s view is 
just the sort of thing I mean as a 
research contribution to the SASS 
design. Whether he is right, he 
asks us to consider a different 
theory of organizations and how it 
might provide insight into what all 
acknowledge as the complex 
interactions that formal education 
represent. 



Final Thoughts 

In the above I have argued for a vision of 
SASS as an integral part of a developing 
national data system in education by being 
a tool for use by state and local 
educational agencies. 8 I have also argued 
that the primary focus should be in 
providing school and staffing data, linked 
to student/school achievement data, most 
useful to state and local policymakers. I 
also have insisted that such a focus dictates 
restraint in thinking about how much of a 
research instrument SASS should be. 

There are many things researchers would 
like to know, that would be important and 
interesting to know, but that are not 
feasible in a survey designed for the 
purposes I have suggested. That the SASS 
design be informed by the best research 
available, and that SASS provide data 
useful to researchers, is without doubt. It 
should not be, in itself, a research 
instrument. 

Nor should SASS be seen as a federal 
instrument to monitor and evaluate federal 
education programs. SASS can provide, 
as it has, data on special populations that 
may be useful for federal and state 
programs, but the focus should always be 
on what is happening fundamentally in the 
schools and how that is related to student 
achievement. In this light it is useful to 
keep in mind Boe’s point about identifying 
genuinely fundamental aspects of schooling 
and keeping these at the center of 
attention. 

Without such a strict focus, SASS may 
become the perennial "Christmas tree," 
resembling more a congressional 
appropriations bill than a serious survey 
instrument. 



On Learning 

Learning is a painful experience 
because we learn from mistakes, which are 
painful, and because in learning we give 
up something of ourselves, some old belief 
or action. Thinking is unpleasant not 
because it demands effort and 
concentration (so does love) but because 
our thoughts come trailing wisps of 
anxiety— we might learn something. It is 
not surprising that alternatives to taking 
thought are attractive, especially if they 
present themselves not as thoughtlessness 
but as conclusions of profound and proven 
reflections. The fallacy embedded in this 
pattern may be called sloganizing . 

Abraham Kaplan 
In Pursuit of Wisdom, p. 146 
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1. Susan Choy has an interesting discussion of "emerging issues” (pp. 215-217) consistent with Boe’s 
guidelines. She rightly reminds us that in addressing such issues real problems of common 
understanding of terms are present. See, for example, her point about what ’tech prep’ means. 

2. There is a special aspect of this issue having to do with the role of researchers as users of NCES data 
that will be discussed later in this paper. 

3. In its initial reports the National Education Goals Panel used statements about how much funding comes 
from federal, state, and local sources for education, citing the average figures. As NCES pointed out, 
these average figures masked a great variation among the states in the level of support from local 
funds. Deep Throat said, "Follow the money." Always sage advice! 

4. Well, to be fair, there is one critical issue, discussed below-linking to student outcomes. 

5. Discussion of Kaufmann’s proposals is contained in McLaughlin, D., Comments on Linking Student 
Data to SASS: Why, When, and How , (unpublished paper), American Institutes for Research, Palo 
Alto, CA. 

6. Ibid. 

7. Ibid. 

8. I have not spoken at all of private education. I do not mean by this omission to imply that there is no 
role for a private sector SASS. Indeed, I would argue that the private system is part of an overall 
public educational system. States charter private schools, and in many states "charter schools" are 
becoming an explicit part of public education. I predict that in the years ahead much more of this sort 
of schooling will emerge. And this will be the result of public policy decisions. Thus, SASS will need 
to be open to this sort of development as well as continuing its more narrowly defined private school 
dimension. 
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